07:08:06,972 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:08:07,534 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-258737476063373368] 07:08:07,558 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 07:08:07,716 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 07:08:07,716 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 07:08:07,812 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:08:07,812 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:08:08,132 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 07:08:08,133 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 07:08:08,142 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 07:08:08,170 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:08,174 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource Mockito is currently self-attaching to enable the inline-mock-maker. This will no longer work in future releases of the JDK. Please add Mockito as an agent to your build what is described in Mockito's documentation: https://javadoc.io/doc/org.mockito/mockito-core/latest/org/mockito/Mockito.html#0.3 WARNING: A Java agent has been loaded dynamically (/tmp/r/net/bytebuddy/byte-buddy-agent/1.15.4/byte-buddy-agent-1.15.4.jar) WARNING: If a serviceability tool is in use, please run with -XX:+EnableDynamicAgentLoading to hide this warning WARNING: If a serviceability tool is not in use, please run with -Djdk.instrument.traceUsage for more information WARNING: Dynamic loading of agents will be disallowed by default in a future release 07:08:09,539 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testBackupDatastore 07:08:09,713 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#2146186115] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:09,722 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 07:08:09,757 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 07:08:09,762 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testBackupDatastore 07:08:09,776 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testBackupDatastore: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testBackupDatastore, generation=0} 07:08:09,818 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testBackupDatastore 07:08:09,822 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:09,822 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore 07:08:09,822 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore 07:08:09,879 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@10bbbd4f} 07:08:09,887 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testBackupDatastore, generation=0} started 07:08:09,893 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:09,893 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testBackupDatastore 07:08:09,896 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore 07:08:09,896 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore to pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore 07:08:09,897 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testBackupDatastore on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore on shard member-1-shard-cars-config_testBackupDatastore whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#113550137] 07:08:09,898 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testBackupDatastore 07:08:09,900 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore 07:08:09,900 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore to pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore 07:08:09,900 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testBackupDatastore on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore on shard member-1-shard-people-config_testBackupDatastore whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore#-257809313] 07:08:09,903 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore#-257809313] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:09,904 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#113550137] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:09,936 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Shard created : member-1-shard-people-config_testBackupDatastore, persistent : true 07:08:09,936 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-config_testBackupDatastore, persistent : true 07:08:09,943 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#2146186115] 07:08:09,958 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:08:09,963 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 07:08:10,14 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#2146186115] 07:08:10,15 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:08:10,15 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 07:08:10,67 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#2146186115] 07:08:10,67 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:08:10,68 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 07:08:10,93 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore-notifier#25455917 created and ready for shard:member-1-shard-cars-config_testBackupDatastore 07:08:10,93 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore-notifier#313362934] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:10,95 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore-notifier#313362934 created and ready for shard:member-1-shard-people-config_testBackupDatastore 07:08:10,96 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore-notifier#25455917] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:10,99 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore/shard-member-1:snapshot-read#-1745561518] 07:08:10,100 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore/shard-member-1:snapshot-read#-1745561518] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:10,100 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore/shard-member-1:snapshot-read#-521450683] 07:08:10,102 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore/shard-member-1:snapshot-read#-521450683] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:10,103 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-1-shard-people-config_testBackupDatastore with journal batch size 1 07:08:10,103 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testBackupDatastore with journal batch size 1 07:08:10,120 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#2146186115] 07:08:10,120 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:08:10,120 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 07:08:10,120 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:10,121 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:10,130 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-config_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:10,132 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 07:08:10,132 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testBackupDatastore is initialized 07:08:10,133 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-config_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:10,133 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 07:08:10,133 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testBackupDatastore is initialized 07:08:10,142 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 07:08:10,142 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:10,142 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 07:08:10,143 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:10,144 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , received role change from null to Follower 07:08:10,153 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , received role change from null to Follower 07:08:10,158 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore (Follower): Received TimeoutNow - switching to Candidate 07:08:10,158 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore (Follower): Received TimeoutNow - switching to Candidate 07:08:10,160 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore (Candidate): Election: Candidate has following voting peers: [] 07:08:10,160 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore (Candidate): Election: Candidate has following voting peers: [] 07:08:10,164 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-config_testBackupDatastore (Candidate): Starting new election term 1 07:08:10,164 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-config_testBackupDatastore (Candidate): Starting new election term 1 07:08:10,164 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-config_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:08:10,164 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-config_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:08:10,164 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 07:08:10,165 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 07:08:10,165 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:10,165 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:10,165 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , received role change from Follower to Candidate 07:08:10,165 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , received role change from Follower to Candidate 07:08:10,171 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#2146186115] 07:08:10,171 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:08:10,172 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#113550137] 07:08:10,189 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore (Candidate): Received ElectionTimeout 07:08:10,191 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore (Leader): Election: Leader has following peers: [] 07:08:10,191 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 07:08:10,191 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 07:08:10,194 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: became leader with frontend state for [] 07:08:10,199 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , received role change from Candidate to Leader 07:08:10,199 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore (Candidate): Received ElectionTimeout 07:08:10,199 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore (Leader): Election: Leader has following peers: [] 07:08:10,199 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-config_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 07:08:10,199 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 07:08:10,199 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: became leader with frontend state for [] 07:08:10,202 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , received role change from Candidate to Leader 07:08:10,203 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , registered listener pekko://cluster-test/user/shardmanager-config_testBackupDatastore 07:08:10,203 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testBackupDatastore, leaderId=member-1-shard-people-config_testBackupDatastore, leaderPayloadVersion=13] 07:08:10,207 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testBackupDatastore from Candidate to Leader 07:08:10,207 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , registered listener pekko://cluster-test/user/shardmanager-config_testBackupDatastore 07:08:10,207 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testBackupDatastore, leaderId=member-1-shard-cars-config_testBackupDatastore, leaderPayloadVersion=13] 07:08:10,207 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testBackupDatastore from Candidate to Leader 07:08:10,207 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testBackupDatastore is ready 07:08:10,258 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#2146186115] 07:08:10,259 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 07:08:10,259 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore#-257809313] 07:08:10,260 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:10,268 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:10,278 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testBackupDatastore 07:08:10,280 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore#-1392289328] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:10,282 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 07:08:10,285 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 07:08:10,285 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testBackupDatastore 07:08:10,286 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testBackupDatastore 07:08:10,286 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:10,286 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore 07:08:10,287 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testBackupDatastore: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testBackupDatastore, generation=0} 07:08:10,287 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore 07:08:10,288 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@577b4e2c} 07:08:10,288 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testBackupDatastore, generation=0} started 07:08:10,289 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:10,289 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testBackupDatastore 07:08:10,291 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Shard created : member-1-shard-cars-oper_testBackupDatastore, persistent : true 07:08:10,292 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testBackupDatastore with journal batch size 1 07:08:10,292 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore/shard-member-1:snapshot-read#-1061841246] 07:08:10,292 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore-notifier#-1787445059 created and ready for shard:member-1-shard-cars-oper_testBackupDatastore 07:08:10,292 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore#-2035599487] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:10,293 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore 07:08:10,293 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore to pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore 07:08:10,295 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testBackupDatastore on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore on shard member-1-shard-cars-oper_testBackupDatastore whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore#-2035599487] 07:08:10,295 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:10,295 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-oper_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:10,295 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 07:08:10,295 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:10,296 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , received role change from null to Follower 07:08:10,295 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testBackupDatastore 07:08:10,297 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore 07:08:10,297 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore to pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore 07:08:10,297 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testBackupDatastore on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore on shard member-1-shard-people-oper_testBackupDatastore whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore#-514055259] 07:08:10,297 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 07:08:10,297 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Shard created : member-1-shard-people-oper_testBackupDatastore, persistent : true 07:08:10,297 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testBackupDatastore is initialized 07:08:10,296 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore (Follower): Received TimeoutNow - switching to Candidate 07:08:10,297 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore (Candidate): Election: Candidate has following voting peers: [] 07:08:10,298 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-oper_testBackupDatastore (Candidate): Starting new election term 1 07:08:10,298 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-oper_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:08:10,298 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 07:08:10,298 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:10,298 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , received role change from Follower to Candidate 07:08:10,299 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore-notifier#-34734580 created and ready for shard:member-1-shard-people-oper_testBackupDatastore 07:08:10,299 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testBackupDatastore with journal batch size 1 07:08:10,299 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore/shard-member-1:snapshot-read#574615064] 07:08:10,299 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore (Candidate): Received ElectionTimeout 07:08:10,300 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore (Leader): Election: Leader has following peers: [] 07:08:10,300 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-oper_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 07:08:10,300 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 07:08:10,300 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: became leader with frontend state for [] 07:08:10,301 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , received role change from Candidate to Leader 07:08:10,301 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , registered listener pekko://cluster-test/user/shardmanager-oper_testBackupDatastore 07:08:10,301 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:10,301 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-oper_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:10,301 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 07:08:10,301 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:10,302 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , received role change from null to Follower 07:08:10,302 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore (Follower): Received TimeoutNow - switching to Candidate 07:08:10,302 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore (Candidate): Election: Candidate has following voting peers: [] 07:08:10,302 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-oper_testBackupDatastore (Candidate): Starting new election term 1 07:08:10,302 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-oper_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:08:10,302 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 07:08:10,302 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:10,302 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , received role change from Follower to Candidate 07:08:10,302 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testBackupDatastore, leaderId=member-1-shard-cars-oper_testBackupDatastore, leaderPayloadVersion=13] 07:08:10,303 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore-notifier#-1787445059] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:10,303 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore (Candidate): Received ElectionTimeout 07:08:10,303 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore (Leader): Election: Leader has following peers: [] 07:08:10,303 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 07:08:10,304 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 07:08:10,304 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: became leader with frontend state for [] 07:08:10,304 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testBackupDatastore from Candidate to Leader 07:08:10,304 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 07:08:10,304 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , received role change from Candidate to Leader 07:08:10,305 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testBackupDatastore is initialized 07:08:10,305 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , registered listener pekko://cluster-test/user/shardmanager-oper_testBackupDatastore 07:08:10,305 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testBackupDatastore, leaderId=member-1-shard-people-oper_testBackupDatastore, leaderPayloadVersion=13] 07:08:10,306 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testBackupDatastore from Candidate to Leader 07:08:10,306 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testBackupDatastore is ready 07:08:10,342 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore#-1392289328] 07:08:10,342 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 07:08:10,343 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore#-2035599487] 07:08:10,394 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore#-1392289328] 07:08:10,394 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 07:08:10,394 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore#-514055259] 07:08:10,420 AM [main] [DEBUG] ClusterAdminRpcService - backupDatastore: BackupDatastoreInput{filePath=target/testBackupDatastore} 07:08:10,423 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: onGetSnapshot 07:08:10,437 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onGetSnapshot 07:08:10,437 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onGetSnapshot 07:08:10,439 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Expecting 2 shard snapshot replies 07:08:10,440 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: onGetSnapshot 07:08:10,441 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onGetSnapshot 07:08:10,441 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 07:08:10,441 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 07:08:10,443 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onGetSnapshot 07:08:10,444 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 07:08:10,441 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Expecting 2 shard snapshot replies 07:08:10,445 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 07:08:10,461 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 07:08:10,464 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 07:08:10,462 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 07:08:10,466 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 07:08:10,465 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Received GetSnapshotReply [id=member-1-shard-cars-config_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@6e7f1c94, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testBackupDatastore, ServerConfigPayload=null]] 07:08:10,466 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply [id=member-1-shard-cars-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@62526f04, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testBackupDatastore, ServerConfigPayload=null]] 07:08:10,481 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply [id=member-1-shard-people-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@2778245a, electionTerm=1, electionVotedFor=member-1-shard-people-oper_testBackupDatastore, ServerConfigPayload=null]] 07:08:10,481 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Received GetSnapshotReply [id=member-1-shard-people-config_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@396dfd61, electionTerm=1, electionVotedFor=member-1-shard-people-config_testBackupDatastore, ServerConfigPayload=null]] 07:08:10,482 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: All shard snapshots received 07:08:10,482 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: All shard snapshots received 07:08:10,532 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] ClusterAdminRpcService - Successfully backed up datastore to file target/testBackupDatastore 07:08:10,562 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#2146186115] 07:08:10,562 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Received unhandled message org.opendaylight.controller.cluster.datastore.DatastoreContext@3267ba70 07:08:10,564 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:08:10,564 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#113550137] 07:08:10,599 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testBackupDatastore 07:08:10,605 AM [main] [DEBUG] ClusterAdminRpcService - backupDatastore: BackupDatastoreInput{filePath=target/testBackupDatastore} 07:08:10,606 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: onGetSnapshot 07:08:10,606 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: onGetSnapshot 07:08:10,607 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Expecting 2 shard snapshot replies 07:08:10,607 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Expecting 2 shard snapshot replies 07:08:10,608 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onGetSnapshot 07:08:10,608 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 07:08:10,609 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 07:08:10,609 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onGetSnapshot 07:08:10,609 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 07:08:10,609 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 07:08:10,610 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply [id=member-1-shard-people-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@285ec366, electionTerm=1, electionVotedFor=member-1-shard-people-oper_testBackupDatastore, ServerConfigPayload=null]] 07:08:10,610 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply [id=member-1-shard-cars-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@523bbbd2, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testBackupDatastore, ServerConfigPayload=null]] 07:08:10,610 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: All shard snapshots received 07:08:10,609 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onGetSnapshot 07:08:10,611 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 07:08:10,611 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 07:08:10,611 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Received GetSnapshotReply [id=member-1-shard-people-config_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@4bb6eac2, electionTerm=1, electionVotedFor=member-1-shard-people-config_testBackupDatastore, ServerConfigPayload=null]] 07:08:10,823 AM [cluster-test-pekko.actor.default-dispatcher-13] [WARN] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Timed out after 200 ms while waiting for snapshot replies from 2 shard(s). 1 shard(s) [cars] did not respond 07:08:10,825 AM [cluster-test-pekko.actor.default-dispatcher-20] [ERROR] ClusterAdminRpcService - Failed to back up datastore to file target/testBackupDatastore java.util.concurrent.TimeoutException: Timed out after 200 ms while waiting for snapshot replies from 2 shard(s). 1 shard(s) [cars] did not respond. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManagerGetSnapshotReplyActor.onReceive(ShardManagerGetSnapshotReplyActor.java:66) at org.apache.pekko.actor.UntypedAbstractActor$$anonfun$receive$1.applyOrElse(AbstractActor.scala:341) at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.apache.pekko.actor.AbstractActor.aroundReceive(AbstractActor.scala:229) at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 07:08:10,836 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testBackupDatastore, generation=0} 07:08:10,839 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testBackupDatastore 07:08:10,840 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testBackupDatastore 07:08:10,840 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onShutDown 07:08:10,840 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 07:08:10,843 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@6f3fa6e0 07:08:10,843 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-config_testBackupDatastore: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@6f3fa6e0 07:08:10,844 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-people-config_testBackupDatastore 07:08:10,845 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 07:08:10,845 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 07:08:10,846 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testBackupDatastore, generation=0} 07:08:10,847 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testBackupDatastore 07:08:10,847 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testBackupDatastore 07:08:10,847 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 07:08:10,847 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onShutDown 07:08:10,847 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@48d63229 07:08:10,847 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testBackupDatastore: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@48d63229 07:08:10,848 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onShutDown 07:08:10,848 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@5cabbd 07:08:10,848 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-people-oper_testBackupDatastore: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@5cabbd 07:08:10,848 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testBackupDatastore 07:08:10,849 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testBackupDatastore 07:08:10,849 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 07:08:10,850 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 07:08:10,854 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:08:10,856 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:08:10,932 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:08:10,960 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [3386858431514065241] 07:08:10,961 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 07:08:10,966 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 07:08:10,966 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 07:08:10,980 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:08:10,981 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:08:11,7 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:11,8 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 07:08:11,8 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 07:08:11,9 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 07:08:11,15 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:11,21 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddReplicasForAllShards 07:08:11,33 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 07:08:11,33 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-496402163] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:11,40 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 07:08:11,40 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testAddReplicasForAllShards 07:08:11,41 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testAddReplicasForAllShards 07:08:11,41 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:11,41 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 07:08:11,42 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 07:08:11,46 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 07:08:11,48 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@3cd9cbf3} 07:08:11,48 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddReplicasForAllShards, generation=0} started 07:08:11,49 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:11,49 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testAddReplicasForAllShards 07:08:11,51 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 07:08:11,51 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 07:08:11,51 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testAddReplicasForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards on shard member-1-shard-cars-config_testAddReplicasForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#1254035363] 07:08:11,51 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testAddReplicasForAllShards 07:08:11,52 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 07:08:11,52 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 07:08:11,52 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testAddReplicasForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards on shard member-1-shard-people-config_testAddReplicasForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#2124651423] 07:08:11,53 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-config_testAddReplicasForAllShards, persistent : true 07:08:11,54 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards/shard-member-1:snapshot-read#1208500400] 07:08:11,54 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards-notifier#-2060950716 created and ready for shard:member-1-shard-cars-config_testAddReplicasForAllShards 07:08:11,55 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testAddReplicasForAllShards with journal batch size 1 07:08:11,55 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-people-config_testAddReplicasForAllShards, persistent : true 07:08:11,56 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards-notifier#-684264575 created and ready for shard:member-1-shard-people-config_testAddReplicasForAllShards 07:08:11,56 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards/shard-member-1:snapshot-read#-1241263250] 07:08:11,57 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#1254035363] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:11,57 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards-notifier#-2060950716] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:11,58 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards/shard-member-1:snapshot-read#1208500400] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:11,58 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#2124651423] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:11,58 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards-notifier#-684264575] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:11,58 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards/shard-member-1:snapshot-read#-1241263250] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:11,59 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-people-config_testAddReplicasForAllShards with journal batch size 1 07:08:11,60 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:11,60 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:11,60 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 07:08:11,60 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testAddReplicasForAllShards is initialized 07:08:11,61 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 07:08:11,61 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:11,61 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from null to Follower 07:08:11,61 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Follower): Received TimeoutNow - switching to Candidate 07:08:11,61 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Candidate): Election: Candidate has following voting peers: [] 07:08:11,62 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Candidate): Starting new election term 1 07:08:11,62 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:08:11,62 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 07:08:11,62 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:11,62 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from Follower to Candidate 07:08:11,63 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Candidate): Received ElectionTimeout 07:08:11,63 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Election: Leader has following peers: [] 07:08:11,63 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 07:08:11,63 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 07:08:11,63 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: became leader with frontend state for [] 07:08:11,63 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from Candidate to Leader 07:08:11,64 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 07:08:11,68 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:11,68 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:11,68 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 07:08:11,68 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testAddReplicasForAllShards is initialized 07:08:11,68 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 07:08:11,68 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:11,68 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Follower): Received TimeoutNow - switching to Candidate 07:08:11,69 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Candidate): Election: Candidate has following voting peers: [] 07:08:11,69 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from null to Follower 07:08:11,69 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Candidate): Starting new election term 1 07:08:11,69 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:08:11,69 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 07:08:11,69 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:11,69 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from Follower to Candidate 07:08:11,70 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Candidate): Received ElectionTimeout 07:08:11,70 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Election: Leader has following peers: [] 07:08:11,70 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 07:08:11,70 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 07:08:11,70 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: became leader with frontend state for [] 07:08:11,70 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from Candidate to Leader 07:08:11,70 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 07:08:11,71 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testAddReplicasForAllShards, leaderId=member-1-shard-people-config_testAddReplicasForAllShards, leaderPayloadVersion=13] 07:08:11,71 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testAddReplicasForAllShards from Candidate to Leader 07:08:11,73 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testAddReplicasForAllShards, leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, leaderPayloadVersion=13] 07:08:11,73 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddReplicasForAllShards from Candidate to Leader 07:08:11,73 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddReplicasForAllShards is ready 07:08:11,99 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-496402163] 07:08:11,99 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:08:11,100 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#1254035363] 07:08:11,150 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-496402163] 07:08:11,151 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 07:08:11,151 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#2124651423] 07:08:11,152 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:11,160 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:11,168 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddReplicasForAllShards 07:08:11,171 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#-2055856628] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:11,172 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 07:08:11,173 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 07:08:11,173 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testAddReplicasForAllShards 07:08:11,174 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testAddReplicasForAllShards 07:08:11,174 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:11,174 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 07:08:11,174 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 07:08:11,174 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 07:08:11,175 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@67b5dbd0} 07:08:11,175 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} started 07:08:11,176 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:11,176 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testAddReplicasForAllShards 07:08:11,177 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 07:08:11,177 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 07:08:11,177 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testAddReplicasForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards on shard member-1-shard-cars-oper_testAddReplicasForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#-1981751169] 07:08:11,177 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testAddReplicasForAllShards 07:08:11,177 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - Shard created : member-1-shard-cars-oper_testAddReplicasForAllShards, persistent : true 07:08:11,178 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#-1981751169] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:11,178 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 07:08:11,178 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 07:08:11,178 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testAddReplicasForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards on shard member-1-shard-people-oper_testAddReplicasForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#1222329906] 07:08:11,178 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-people-oper_testAddReplicasForAllShards, persistent : true 07:08:11,179 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testAddReplicasForAllShards with journal batch size 1 07:08:11,179 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards-notifier#-1538558149 created and ready for shard:member-1-shard-cars-oper_testAddReplicasForAllShards 07:08:11,179 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards/shard-member-1:snapshot-read#-556747616] 07:08:11,179 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testAddReplicasForAllShards with journal batch size 1 07:08:11,180 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards/shard-member-1:snapshot-read#1179456226] 07:08:11,181 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:11,181 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:11,181 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:11,181 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:11,182 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 07:08:11,182 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:11,182 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 07:08:11,182 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 07:08:11,182 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testAddReplicasForAllShards is initialized 07:08:11,182 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:11,182 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 07:08:11,182 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testAddReplicasForAllShards is initialized 07:08:11,182 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Follower): Received TimeoutNow - switching to Candidate 07:08:11,182 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Candidate): Election: Candidate has following voting peers: [] 07:08:11,182 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Candidate): Starting new election term 1 07:08:11,182 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:08:11,182 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 07:08:11,182 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:11,182 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from null to Follower 07:08:11,183 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Candidate): Received ElectionTimeout 07:08:11,183 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Election: Leader has following peers: [] 07:08:11,184 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 07:08:11,184 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 07:08:11,184 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: became leader with frontend state for [] 07:08:11,187 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#1222329906] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:11,191 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards-notifier#1217915311 created and ready for shard:member-1-shard-people-oper_testAddReplicasForAllShards 07:08:11,196 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from null to Follower 07:08:11,197 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from Follower to Candidate 07:08:11,191 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Follower): Received TimeoutNow - switching to Candidate 07:08:11,197 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from Candidate to Leader 07:08:11,197 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Candidate): Election: Candidate has following voting peers: [] 07:08:11,197 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 07:08:11,197 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Candidate): Starting new election term 1 07:08:11,197 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:08:11,197 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 07:08:11,197 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:11,197 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from Follower to Candidate 07:08:11,197 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testAddReplicasForAllShards, leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, leaderPayloadVersion=13] 07:08:11,198 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testAddReplicasForAllShards from Candidate to Leader 07:08:11,198 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Candidate): Received ElectionTimeout 07:08:11,199 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Election: Leader has following peers: [] 07:08:11,199 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 07:08:11,199 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 07:08:11,199 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: became leader with frontend state for [] 07:08:11,199 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from Candidate to Leader 07:08:11,199 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 07:08:11,199 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testAddReplicasForAllShards, leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, leaderPayloadVersion=13] 07:08:11,200 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddReplicasForAllShards from Candidate to Leader 07:08:11,200 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddReplicasForAllShards is ready 07:08:11,226 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#-2055856628] 07:08:11,226 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 07:08:11,226 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#-1981751169] 07:08:11,277 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#-2055856628] 07:08:11,277 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 07:08:11,277 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#1222329906] 07:08:11,294 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@5bfc300f] 07:08:11,302 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1 doCreateShard: shardId: member-1-shard-pets-config_testAddReplicasForAllShards, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: true 07:08:11,304 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - Shard created : member-1-shard-pets-config_testAddReplicasForAllShards, persistent : true 07:08:11,306 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards-notifier#-1680749593 created and ready for shard:member-1-shard-pets-config_testAddReplicasForAllShards 07:08:11,307 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards/shard-member-1:snapshot-read#-191962251] 07:08:11,306 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - Starting recovery for member-1-shard-pets-config_testAddReplicasForAllShards with journal batch size 1 07:08:11,308 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:11,308 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:11,308 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 07:08:11,308 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:11,308 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , received role change from null to Follower 07:08:11,309 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Follower): Received TimeoutNow - switching to Candidate 07:08:11,309 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Candidate): Election: Candidate has following voting peers: [] 07:08:11,309 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Candidate): Starting new election term 1 07:08:11,309 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:08:11,309 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 07:08:11,309 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:11,309 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , received role change from Follower to Candidate 07:08:11,310 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Candidate): Received ElectionTimeout 07:08:11,310 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Election: Leader has following peers: [] 07:08:11,310 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 07:08:11,310 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 07:08:11,310 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: became leader with frontend state for [] 07:08:11,310 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , received role change from Candidate to Leader 07:08:11,311 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [pets] 07:08:11,311 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-pets-config_testAddReplicasForAllShards is initialized 07:08:11,311 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 07:08:11,312 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-pets-config_testAddReplicasForAllShards, leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, leaderPayloadVersion=13] 07:08:11,312 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testAddReplicasForAllShards from Candidate to Leader 07:08:11,312 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddReplicasForAllShards is ready 07:08:11,361 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-496402163] 07:08:11,361 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : pets 07:08:11,362 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#1993622414] 07:08:11,379 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:08:11,403 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [5178423692212300979] 07:08:11,404 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 07:08:11,413 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:08:11,414 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 07:08:11,416 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:08:11,416 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:08:11,447 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:11,451 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:11,459 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddReplicasForAllShards 07:08:11,479 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-1074180417] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:11,480 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 07:08:11,492 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 07:08:11,493 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 07:08:11,495 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@521362b7} 07:08:11,495 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddReplicasForAllShards, generation=0} started 07:08:11,496 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:11,496 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:11,499 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:11,503 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddReplicasForAllShards 07:08:11,506 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 07:08:11,506 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#2123957484] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:11,509 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 07:08:11,510 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 07:08:11,511 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@5cd49685} 07:08:11,516 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} started 07:08:11,517 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:11,833 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 07:08:11,886 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 07:08:11,887 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:11,887 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddReplicasForAllShards is ready 07:08:11,887 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:11,887 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 07:08:12,6 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 07:08:12,7 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:12,7 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-pets-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards 07:08:12,7 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-pets-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards 07:08:12,7 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards 07:08:12,7 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 07:08:12,7 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards 07:08:12,7 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 07:08:12,7 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddReplicasForAllShards is ready 07:08:12,7 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:12,7 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards 07:08:12,7 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 07:08:12,8 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards 07:08:12,8 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 07:08:12,8 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddReplicasForAllShards is ready 07:08:12,34 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@58eb09c7] 07:08:12,36 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2 doCreateShard: shardId: member-2-shard-pets-config_testAddReplicasForAllShards, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: false 07:08:12,38 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-2-shard-pets-config_testAddReplicasForAllShards, persistent : true 07:08:12,38 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards#-810230011] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:12,40 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-2-shard-pets-config_testAddReplicasForAllShards with journal batch size 1 07:08:12,40 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=no-leader-ns, moduleName=no-leader-module, shardName=no-leader, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@4a28a37] 07:08:12,41 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2 doCreateShard: shardId: member-2-shard-no-leader-oper_testAddReplicasForAllShards, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: false 07:08:12,42 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-notifier#1254250711 created and ready for shard:member-2-shard-pets-config_testAddReplicasForAllShards 07:08:12,42 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-notifier#1254250711] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:12,42 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/shard-member-2:snapshot-read#-201711934] 07:08:12,43 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/shard-member-2:snapshot-read#-201711934] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:12,44 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards#1325582489] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:12,44 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - Shard created : member-2-shard-no-leader-oper_testAddReplicasForAllShards, persistent : true 07:08:12,45 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards-notifier#1712692948 created and ready for shard:member-2-shard-no-leader-oper_testAddReplicasForAllShards 07:08:12,45 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards-notifier#1712692948] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:12,45 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards/shard-member-2:snapshot-read#-760066260] 07:08:12,46 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards/shard-member-2:snapshot-read#-760066260] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:12,46 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - Starting recovery for member-2-shard-no-leader-oper_testAddReplicasForAllShards with journal batch size 1 07:08:12,47 AM [main] [INFO] ClusterAdminRpcService - Adding replicas for all shards 07:08:12,47 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people, pets] for data store config_testAddReplicasForAllShards 07:08:12,48 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:12,48 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 07:08:12,48 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:12,48 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 07:08:12,48 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people, no-leader] for data store oper_testAddReplicasForAllShards 07:08:12,48 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 07:08:12,48 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:12,49 AM [cluster-test-shard-dispatcher-12] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 07:08:12,49 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:12,49 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:12,50 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-no-leader-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 07:08:12,50 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:12,50 AM [cluster-test-shard-dispatcher-12] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 07:08:12,50 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testAddReplicasForAllShards , received role change from null to Follower 07:08:12,50 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-no-leader-oper_testAddReplicasForAllShards , received role change from null to Follower 07:08:12,58 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=people] 07:08:12,58 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=no-leader] 07:08:12,58 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [no-leader] 07:08:12,58 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Shard member-2-shard-no-leader-oper_testAddReplicasForAllShards is initialized 07:08:12,58 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 07:08:12,60 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-no-leader-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 07:08:12,60 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards] 07:08:12,61 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 07:08:12,61 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for people forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards] 07:08:12,61 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=no-leader, waitUntilReady=true] 07:08:12,61 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for no-leader forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards] 07:08:12,61 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-no-leader-oper_testAddReplicasForAllShards from null to Follower 07:08:12,67 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=people] 07:08:12,67 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=pets] 07:08:12,68 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [pets] 07:08:12,68 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-2-shard-pets-config_testAddReplicasForAllShards is initialized 07:08:12,68 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 07:08:12,68 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards] 07:08:12,68 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 07:08:12,68 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 07:08:12,68 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for people forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards] 07:08:12,69 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=pets, waitUntilReady=true] 07:08:12,69 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for pets forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards] 07:08:12,69 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-pets-config_testAddReplicasForAllShards from null to Follower 07:08:12,75 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 07:08:12,76 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#-1981751169, primaryVersion=13] 07:08:12,76 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=people, waitUntilReady=true] 07:08:12,77 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for people: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#1222329906, primaryVersion=13] 07:08:12,77 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=no-leader, waitUntilReady=true] 07:08:12,77 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: No shard found for no-leader 07:08:12,77 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 07:08:12,77 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#1254035363, primaryVersion=13] 07:08:12,78 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=people, waitUntilReady=true] 07:08:12,78 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for people: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#2124651423, primaryVersion=13] 07:08:12,78 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=pets, waitUntilReady=true] 07:08:12,78 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for pets: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#1993622414, primaryVersion=13] 07:08:12,89 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#-1981751169 for shard member-2-shard-cars-oper_testAddReplicasForAllShards 07:08:12,92 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards#-242761008] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:12,93 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - Shard created : member-2-shard-cars-oper_testAddReplicasForAllShards, persistent : true 07:08:12,93 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Failed to find leader for shard no-leader: received response: {} org.opendaylight.controller.cluster.datastore.exceptions.PrimaryNotFoundException: No primary shard found for no-leader. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.findPrimary(ShardManager.java:1056) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:211) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 07:08:12,93 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/shard-member-2:snapshot-read#-1119343728] 07:08:12,94 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#1254035363 for shard member-2-shard-cars-config_testAddReplicasForAllShards 07:08:12,94 AM [cluster-test-pekko.actor.default-dispatcher-11] [WARN] ClusterAdminRpcService - Failed to add replica for shard no-leader, type Operational org.opendaylight.controller.cluster.datastore.exceptions.PrimaryNotFoundException: No primary shard found for no-leader. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.findPrimary(ShardManager.java:1056) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:211) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 07:08:12,94 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/shard-member-2:snapshot-read#-1119343728] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:12,95 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-2-shard-cars-config_testAddReplicasForAllShards, persistent : true 07:08:12,95 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#2124651423 for shard member-2-shard-people-config_testAddReplicasForAllShards 07:08:12,95 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testAddReplicasForAllShards with journal batch size 1 07:08:12,95 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#1993622414 for shard member-2-shard-pets-config_testAddReplicasForAllShards 07:08:12,96 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-2-shard-people-config_testAddReplicasForAllShards, persistent : true 07:08:12,97 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-2-shard-people-config_testAddReplicasForAllShards with journal batch size 1 07:08:12,97 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-notifier#199141413 created and ready for shard:member-2-shard-cars-oper_testAddReplicasForAllShards 07:08:12,98 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards-notifier#214746343 created and ready for shard:member-2-shard-people-config_testAddReplicasForAllShards 07:08:12,98 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 1 07:08:12,98 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/shard-member-2:snapshot-read#1378895434] 07:08:12,98 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onAddServer: AddServer [newServerId=member-2-shard-cars-oper_testAddReplicasForAllShards, newServerAddress=null, votingMember=true], state: Idle 07:08:12,100 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:12,100 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:12,100 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:12,100 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:12,100 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 07:08:12,100 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 07:08:12,100 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:12,100 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:12,100 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 07:08:12,100 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testAddReplicasForAllShards is initialized 07:08:12,100 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 07:08:12,100 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testAddReplicasForAllShards is initialized 07:08:12,101 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testAddReplicasForAllShards , received role change from null to Follower 07:08:12,101 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 07:08:12,101 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testAddReplicasForAllShards from null to Follower 07:08:12,101 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-notifier#714744052 created and ready for shard:member-2-shard-cars-config_testAddReplicasForAllShards 07:08:12,101 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddReplicasForAllShards , received role change from null to Follower 07:08:12,101 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 07:08:12,101 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testAddReplicasForAllShards from null to Follower 07:08:12,101 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testAddReplicasForAllShards with journal batch size 1 07:08:12,103 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#1222329906 for shard member-2-shard-people-oper_testAddReplicasForAllShards 07:08:12,104 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:12,104 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:12,104 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 07:08:12,104 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testAddReplicasForAllShards is initialized 07:08:12,104 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - Shard created : member-2-shard-people-oper_testAddReplicasForAllShards, persistent : true 07:08:12,105 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: Initiating AddServer [newServerId=member-2-shard-cars-oper_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 07:08:12,105 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/shard-member-2:snapshot-read#2024587038] 07:08:12,104 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 07:08:12,105 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:12,106 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testAddReplicasForAllShards with journal batch size 1 07:08:12,106 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 07:08:12,110 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onAddServer: AddServer [newServerId=member-2-shard-cars-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true], state: Idle 07:08:12,110 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: onAddServer: AddServer [newServerId=member-2-shard-people-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true], state: Idle 07:08:12,110 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onAddServer: AddServer [newServerId=member-2-shard-pets-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true], state: Idle 07:08:12,110 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: Initiating AddServer [newServerId=member-2-shard-people-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 07:08:12,110 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: Initiating AddServer [newServerId=member-2-shard-pets-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 07:08:12,110 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 07:08:12,110 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 07:08:12,110 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onAddServer: AddServer [newServerId=member-2-shard-people-oper_testAddReplicasForAllShards, newServerAddress=null, votingMember=true], state: Idle 07:08:12,110 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: Initiating AddServer [newServerId=member-2-shard-people-oper_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 07:08:12,110 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/shard-member-2:snapshot-read#1849067850] 07:08:12,110 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 07:08:12,110 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: Initiating AddServer [newServerId=member-2-shard-cars-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 07:08:12,111 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 07:08:12,111 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards-notifier#1587649966 created and ready for shard:member-2-shard-people-oper_testAddReplicasForAllShards 07:08:12,111 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddReplicasForAllShards , received role change from null to Follower 07:08:12,111 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 07:08:12,111 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testAddReplicasForAllShards from null to Follower 07:08:12,112 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:12,112 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:12,112 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 07:08:12,112 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:12,112 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testAddReplicasForAllShards , received role change from null to Follower 07:08:12,112 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 07:08:12,112 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testAddReplicasForAllShards is initialized 07:08:12,112 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 07:08:12,113 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testAddReplicasForAllShards from null to Follower 07:08:12,113 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-people-config_testAddReplicasForAllShards 07:08:12,116 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: lastSequenceNumber prior to capture: 1 07:08:12,113 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-oper_testAddReplicasForAllShards 07:08:12,117 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: lastSequenceNumber prior to capture: 1 07:08:12,113 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-pets-config_testAddReplicasForAllShards 07:08:12,117 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: lastSequenceNumber prior to capture: 1 07:08:12,113 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-people-oper_testAddReplicasForAllShards 07:08:12,117 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: lastSequenceNumber prior to capture: 1 07:08:12,112 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-config_testAddReplicasForAllShards 07:08:12,117 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: lastSequenceNumber prior to capture: 1 07:08:12,117 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 07:08:12,118 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 07:08:12,117 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 07:08:12,128 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: Initiating capture snapshot for new server member-2-shard-cars-config_testAddReplicasForAllShards 07:08:12,128 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 07:08:12,129 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: Initiating capture snapshot for new server member-2-shard-people-oper_testAddReplicasForAllShards 07:08:12,129 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: CaptureSnapshotReply received by actor 07:08:12,130 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@352e60b1, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testAddReplicasForAllShards, ServerConfigPayload=null] 07:08:12,130 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 07:08:12,125 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: Initiating capture snapshot for new server member-2-shard-people-config_testAddReplicasForAllShards 07:08:12,127 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: Initiating capture snapshot for new server member-2-shard-cars-oper_testAddReplicasForAllShards 07:08:12,134 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: CaptureSnapshotReply received by actor 07:08:12,134 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: CaptureSnapshotReply received by actor 07:08:12,134 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@3dbe631c, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testAddReplicasForAllShards, ServerConfigPayload=null] 07:08:12,134 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 07:08:12,134 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@349d34c3, electionTerm=1, electionVotedFor=member-1-shard-people-config_testAddReplicasForAllShards, ServerConfigPayload=null] 07:08:12,135 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 07:08:12,127 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 07:08:12,135 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: Initiating capture snapshot for new server member-2-shard-pets-config_testAddReplicasForAllShards 07:08:12,135 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: CaptureSnapshotReply received by actor 07:08:12,135 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@4dae209, electionTerm=1, electionVotedFor=member-1-shard-people-oper_testAddReplicasForAllShards, ServerConfigPayload=null] 07:08:12,135 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 07:08:12,137 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): sendInstallSnapshot 07:08:12,137 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): sendInstallSnapshot 07:08:12,138 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): next snapshot chunk size for follower member-2-shard-people-oper_testAddReplicasForAllShards: 229 07:08:12,138 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): next snapshot chunk size for follower member-2-shard-people-config_testAddReplicasForAllShards: 229 07:08:12,139 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): sendInstallSnapshot 07:08:12,139 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): next snapshot chunk size for follower member-2-shard-cars-oper_testAddReplicasForAllShards: 229 07:08:12,139 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testAddReplicasForAllShards, member-2-shard-people-config_testAddReplicasForAllShards), Chunk: 1/1 07:08:12,139 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-oper_testAddReplicasForAllShards, member-2-shard-cars-oper_testAddReplicasForAllShards), Chunk: 1/1 07:08:12,140 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 07:08:12,140 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Snapshot success - sequence number: -1 07:08:12,140 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 07:08:12,140 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Snapshot success - sequence number: -1 07:08:12,142 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: SnapshotComplete received 07:08:12,143 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: handled a snapshot response 07:08:12,143 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: CaptureSnapshotReply received by actor 07:08:12,143 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@58c6f627, electionTerm=1, electionVotedFor=member-1-shard-pets-config_testAddReplicasForAllShards, ServerConfigPayload=null] 07:08:12,143 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 07:08:12,143 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): sendInstallSnapshot 07:08:12,143 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): next snapshot chunk size for follower member-2-shard-pets-config_testAddReplicasForAllShards: 229 07:08:12,143 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testAddReplicasForAllShards, member-2-shard-pets-config_testAddReplicasForAllShards), Chunk: 1/1 07:08:12,145 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 07:08:12,145 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Snapshot success - sequence number: -1 07:08:12,145 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: SnapshotComplete received 07:08:12,146 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: SnapshotComplete received 07:08:12,148 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: handled a journal response 07:08:12,149 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: handled a journal response 07:08:12,149 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: handled a snapshot response 07:08:12,149 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: handled a journal response 07:08:12,149 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: handled a snapshot response 07:08:12,149 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-oper_testAddReplicasForAllShards, member-2-shard-people-oper_testAddReplicasForAllShards), Chunk: 1/1 07:08:12,149 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 07:08:12,149 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Snapshot success - sequence number: -1 07:08:12,150 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: SnapshotComplete received 07:08:12,150 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): sendInstallSnapshot 07:08:12,150 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): next snapshot chunk size for follower member-2-shard-cars-config_testAddReplicasForAllShards: 229 07:08:12,150 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testAddReplicasForAllShards, member-2-shard-cars-config_testAddReplicasForAllShards), Chunk: 1/1 07:08:12,150 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 07:08:12,150 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Snapshot success - sequence number: -1 07:08:12,151 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: handled a snapshot response 07:08:12,151 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: handled a journal response 07:08:12,153 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 07:08:12,155 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 07:08:12,155 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): handleInstallSnapshot: InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]} 07:08:12,155 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): handleInstallSnapshot: InstallSnapshot{term=1, leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]} 07:08:12,155 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 07:08:12,155 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): handleInstallSnapshot: InstallSnapshot{term=1, leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]} 07:08:12,155 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 07:08:12,155 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): handleInstallSnapshot: InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]} 07:08:12,153 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-people-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 07:08:12,156 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): handleInstallSnapshot: InstallSnapshot{term=1, leaderId=member-1-shard-people-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]} 07:08:12,155 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: SnapshotComplete received 07:08:12,157 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: handled a snapshot response 07:08:12,157 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: handled a journal response 07:08:12,157 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 07:08:12,157 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 07:08:12,158 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Snapshot received from leader: member-1-shard-pets-config_testAddReplicasForAllShards 07:08:12,159 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 07:08:12,159 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Snapshot received from leader: member-1-shard-people-oper_testAddReplicasForAllShards 07:08:12,159 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 07:08:12,159 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Snapshot received from leader: member-1-shard-cars-config_testAddReplicasForAllShards 07:08:12,159 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testAddReplicasForAllShards, leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, leaderPayloadVersion=-1] 07:08:12,160 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-pets-config_testAddReplicasForAllShards, leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, leaderPayloadVersion=-1] 07:08:12,160 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testAddReplicasForAllShards, leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, leaderPayloadVersion=-1] 07:08:12,160 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 07:08:12,160 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Snapshot received from leader: member-1-shard-people-config_testAddReplicasForAllShards 07:08:12,161 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testAddReplicasForAllShards, leaderId=member-1-shard-people-config_testAddReplicasForAllShards, leaderPayloadVersion=-1] 07:08:12,162 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 07:08:12,162 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Applying snapshot on follower: org.opendaylight.controller.cluster.io.FileBackedOutputStream$1@3f1a5257 07:08:12,162 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testAddReplicasForAllShards status sync done false 07:08:12,162 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testAddReplicasForAllShards status sync done false 07:08:12,163 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Snapshot received from leader: member-1-shard-cars-oper_testAddReplicasForAllShards 07:08:12,163 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Applying snapshot on follower: org.opendaylight.controller.cluster.io.FileBackedOutputStream$1@112c3109 07:08:12,163 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testAddReplicasForAllShards status sync done false 07:08:12,163 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Converted InstallSnapshot from leader: member-1-shard-people-config_testAddReplicasForAllShards to state 07:08:12,163 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 07:08:12,164 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 07:08:12,164 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Snapshot success - sequence number: 1 07:08:12,164 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Moving last applied index from -1 to -1 07:08:12,164 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testAddReplicasForAllShards, leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, leaderPayloadVersion=-1] 07:08:12,164 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Applying snapshot on follower: org.opendaylight.controller.cluster.io.FileBackedOutputStream$1@1bd45a79 07:08:12,164 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Applying snapshot 07:08:12,165 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Converted InstallSnapshot from leader: member-1-shard-cars-oper_testAddReplicasForAllShards to state 07:08:12,165 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 07:08:12,165 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendMetadata - member-2-shard-people-config_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 07:08:12,165 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Converted InstallSnapshot from leader: member-1-shard-pets-config_testAddReplicasForAllShards to state 07:08:12,165 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 07:08:12,165 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 07:08:12,165 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Snapshot success - sequence number: 1 07:08:12,165 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Moving last applied index from -1 to -1 07:08:12,165 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Applying snapshot 07:08:12,165 AM [cluster-test-shard-dispatcher-10] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 07:08:12,165 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddReplicasForAllShards status sync done false 07:08:12,166 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 07:08:12,166 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Snapshot success - sequence number: 1 07:08:12,166 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Moving last applied index from -1 to -1 07:08:12,166 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Applying snapshot 07:08:12,166 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-pets-config_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 07:08:12,166 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Applying snapshot on follower: org.opendaylight.controller.cluster.io.FileBackedOutputStream$1@5d49f4aa 07:08:12,166 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Converted InstallSnapshot from leader: member-1-shard-people-oper_testAddReplicasForAllShards to state 07:08:12,166 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 07:08:12,167 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 07:08:12,167 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Snapshot success - sequence number: 1 07:08:12,167 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Moving last applied index from -1 to -1 07:08:12,167 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Applying snapshot 07:08:12,167 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-2-shard-people-oper_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 07:08:12,168 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Applying snapshot on follower: org.opendaylight.controller.cluster.io.FileBackedOutputStream$1@68edd9dc 07:08:12,168 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Converted InstallSnapshot from leader: member-1-shard-cars-config_testAddReplicasForAllShards to state 07:08:12,168 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 07:08:12,169 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 07:08:12,169 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Snapshot success - sequence number: 1 07:08:12,169 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Moving last applied index from -1 to -1 07:08:12,169 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Applying snapshot 07:08:12,169 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 07:08:12,168 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddReplicasForAllShards status sync done false 07:08:12,193 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher#563756782] 07:08:12,213 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-config_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher#-459084480] with name member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher 07:08:12,214 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-oper_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher#896133704] with name member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher 07:08:12,213 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-config_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher#1057118393] with name member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher 07:08:12,213 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-pets-config_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher#563756782] with name member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher 07:08:12,213 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher#-459084480] 07:08:12,215 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher#1057118393] 07:08:12,215 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher#896133704] 07:08:12,215 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-oper_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher#1236325631] with name member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher 07:08:12,216 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher#1236325631] 07:08:12,221 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-config_testAddReplicasForAllShards: publishChanges: rootPath: / 07:08:12,219 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-2-shard-people-config_testAddReplicasForAllShards: state snapshot applied in 54.48 ms 07:08:12,221 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Done applying snapshot 07:08:12,221 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): handleInstallSnapshot returning: InstallSnapshotReply{term=1, followerId=member-2-shard-people-config_testAddReplicasForAllShards, chunkIndex=1, success=true} 07:08:12,222 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-pets-config_testAddReplicasForAllShards: state snapshot applied in 56.03 ms 07:08:12,222 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Done applying snapshot 07:08:12,222 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): handleInstallSnapshot returning: InstallSnapshotReply{term=1, followerId=member-2-shard-pets-config_testAddReplicasForAllShards, chunkIndex=1, success=true} 07:08:12,222 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: SnapshotComplete received 07:08:12,223 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: handled a snapshot response 07:08:12,223 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: handled a journal response 07:08:12,223 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-config_testAddReplicasForAllShards: state snapshot applied in 54.67 ms 07:08:12,223 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Done applying snapshot 07:08:12,223 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): handleInstallSnapshot returning: InstallSnapshotReply{term=1, followerId=member-2-shard-cars-config_testAddReplicasForAllShards, chunkIndex=1, success=true} 07:08:12,224 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: SnapshotComplete received 07:08:12,224 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-pets-config_testAddReplicasForAllShards: publishChanges: rootPath: / 07:08:12,224 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testAddReplicasForAllShards: publishChanges: rootPath: / 07:08:12,224 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: handled a snapshot response 07:08:12,224 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: handled a journal response 07:08:12,224 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testAddReplicasForAllShards: state snapshot applied in 59.11 ms 07:08:12,224 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Done applying snapshot 07:08:12,225 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): handleInstallSnapshot returning: InstallSnapshotReply{term=1, followerId=member-2-shard-cars-oper_testAddReplicasForAllShards, chunkIndex=1, success=true} 07:08:12,225 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testAddReplicasForAllShards: publishChanges: rootPath: / 07:08:12,225 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-oper_testAddReplicasForAllShards: publishChanges: rootPath: / 07:08:12,225 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-2-shard-people-oper_testAddReplicasForAllShards: state snapshot applied in 58.25 ms 07:08:12,225 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Done applying snapshot 07:08:12,225 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): handleInstallSnapshot returning: InstallSnapshotReply{term=1, followerId=member-2-shard-people-oper_testAddReplicasForAllShards, chunkIndex=1, success=true} 07:08:12,225 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: SnapshotComplete received 07:08:12,226 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: SnapshotComplete received 07:08:12,226 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: handled a snapshot response 07:08:12,226 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: SnapshotComplete received 07:08:12,226 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: handled a snapshot response 07:08:12,226 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: handled a snapshot response 07:08:12,226 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: handled a journal response 07:08:12,226 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: handled a journal response 07:08:12,226 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: handled a journal response 07:08:12,228 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-oper_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher: 2.716 ms 07:08:12,228 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-pets-config_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher: 4.069 ms 07:08:12,228 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher: 3.173 ms 07:08:12,228 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher: 4.155 ms 07:08:12,228 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-config_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher: 7.784 ms 07:08:12,230 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): handleInstallSnapshotReply: InstallSnapshotReply{term=1, followerId=member-2-shard-people-config_testAddReplicasForAllShards, chunkIndex=1, success=true} 07:08:12,230 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-people-config_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 07:08:12,230 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 07:08:12,230 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-people-config_testAddReplicasForAllShards] 07:08:12,231 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): handleInstallSnapshotReply: InstallSnapshotReply{term=1, followerId=member-2-shard-cars-config_testAddReplicasForAllShards, chunkIndex=1, success=true} 07:08:12,231 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-cars-config_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 07:08:12,231 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 07:08:12,231 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-cars-config_testAddReplicasForAllShards] 07:08:12,231 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): handleInstallSnapshotReply: InstallSnapshotReply{term=1, followerId=member-2-shard-cars-oper_testAddReplicasForAllShards, chunkIndex=1, success=true} 07:08:12,231 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-cars-oper_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 07:08:12,231 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 07:08:12,231 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-cars-oper_testAddReplicasForAllShards] 07:08:12,231 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): handleInstallSnapshotReply: InstallSnapshotReply{term=1, followerId=member-2-shard-pets-config_testAddReplicasForAllShards, chunkIndex=1, success=true} 07:08:12,232 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-pets-config_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 07:08:12,232 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 07:08:12,232 AM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-pets-config_testAddReplicasForAllShards] 07:08:12,232 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): handleInstallSnapshotReply: InstallSnapshotReply{term=1, followerId=member-2-shard-people-oper_testAddReplicasForAllShards, chunkIndex=1, success=true} 07:08:12,232 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-people-oper_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 07:08:12,232 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 07:08:12,232 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-people-oper_testAddReplicasForAllShards] 07:08:12,232 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]] 07:08:12,232 AM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]] 07:08:12,233 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]] 07:08:12,232 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]] 07:08:12,232 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]] 07:08:12,235 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]] 07:08:12,236 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]] 07:08:12,236 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]] 07:08:12,235 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]] 07:08:12,239 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]] 07:08:12,239 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]] 07:08:12,242 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]] 07:08:12,244 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=9854b3d8-00c1-411f-a8d5-4c6af4e47072}, logIndex: 0, isSendImmediate: true 07:08:12,244 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=e47e3644-3bef-4adf-81ba-d2914f25d62e}, logIndex: 0, isSendImmediate: true 07:08:12,245 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddReplicasForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:12,245 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testAddReplicasForAllShards 07:08:12,242 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=1f6db45b-38c5-4315-923a-27919a7b8e05}, logIndex: 0, isSendImmediate: true 07:08:12,235 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]] 07:08:12,239 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]] 07:08:12,245 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testAddReplicasForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:12,245 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]] 07:08:12,245 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-oper_testAddReplicasForAllShards 07:08:12,245 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testAddReplicasForAllShards: AppendEntries{term=1, leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]]]} 07:08:12,245 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=acbf484c-cbfa-4f13-9c8a-86457025f654}, logIndex: 0, isSendImmediate: true 07:08:12,245 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=f33e5542-b546-4360-9a92-0793392bfe5c}, logIndex: 0, isSendImmediate: true 07:08:12,246 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddReplicasForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:12,246 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testAddReplicasForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:12,246 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-config_testAddReplicasForAllShards 07:08:12,246 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Sending AppendEntries to follower member-2-shard-people-config_testAddReplicasForAllShards: AppendEntries{term=1, leaderId=member-1-shard-people-config_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]]]} 07:08:12,246 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: Returning OK for operation AddServer [newServerId=member-2-shard-cars-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 07:08:12,245 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testAddReplicasForAllShards: AppendEntries{term=1, leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]]]} 07:08:12,246 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-oper_testAddReplicasForAllShards 07:08:12,247 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testAddReplicasForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:12,247 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-pets-config_testAddReplicasForAllShards 07:08:12,247 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testAddReplicasForAllShards: AppendEntries{term=1, leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]]]} 07:08:12,247 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Sending AppendEntries to follower member-2-shard-pets-config_testAddReplicasForAllShards: AppendEntries{term=1, leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]]]} 07:08:12,247 AM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: Returning OK for operation AddServer [newServerId=member-2-shard-pets-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 07:08:12,247 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: Returning OK for operation AddServer [newServerId=member-2-shard-cars-oper_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 07:08:12,247 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: Returning OK for operation AddServer [newServerId=member-2-shard-people-oper_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 07:08:12,255 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: Returning OK for operation AddServer [newServerId=member-2-shard-people-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 07:08:12,256 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]] 07:08:12,257 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]] 07:08:12,257 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]] 07:08:12,257 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]] 07:08:12,256 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]] 07:08:12,259 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]]]} 07:08:12,260 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Number of entries to be appended = 1 07:08:12,260 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 07:08:12,260 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]] 07:08:12,260 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]] 07:08:12,261 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testAddReplicasForAllShards, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards, votingState=VOTING]] 07:08:12,261 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Log size is now 1 07:08:12,261 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=member-2-shard-cars-config_testAddReplicasForAllShards, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:12,262 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=member-1-shard-people-config_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]]]} 07:08:12,262 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Number of entries to be appended = 1 07:08:12,262 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 07:08:12,262 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]] 07:08:12,262 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]] 07:08:12,262 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-config_testAddReplicasForAllShards, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards, votingState=VOTING]] 07:08:12,262 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Log size is now 1 07:08:12,262 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=member-2-shard-people-config_testAddReplicasForAllShards, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:12,262 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]]]} 07:08:12,262 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Number of entries to be appended = 1 07:08:12,262 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 07:08:12,262 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]] 07:08:12,262 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]] 07:08:12,263 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]]]} 07:08:12,263 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testAddReplicasForAllShards} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#-1981751169 07:08:12,263 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Number of entries to be appended = 1 07:08:12,263 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard cars 07:08:12,263 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 07:08:12,263 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]] 07:08:12,263 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]] 07:08:12,263 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testAddReplicasForAllShards, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards, votingState=VOTING]] 07:08:12,263 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Log size is now 1 07:08:12,263 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=member-2-shard-cars-oper_testAddReplicasForAllShards, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:12,263 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]]]} 07:08:12,263 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Number of entries to be appended = 1 07:08:12,263 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 07:08:12,263 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]] 07:08:12,263 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]] 07:08:12,264 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-oper_testAddReplicasForAllShards, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards, votingState=VOTING]] 07:08:12,264 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Log size is now 1 07:08:12,264 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-oper_testAddReplicasForAllShards 07:08:12,264 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=member-2-shard-people-oper_testAddReplicasForAllShards, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:12,264 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars] 07:08:12,264 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testAddReplicasForAllShards, leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, leaderPayloadVersion=13] 07:08:12,264 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 07:08:12,264 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-people-oper_testAddReplicasForAllShards} for shard people from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#1222329906 07:08:12,264 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard people 07:08:12,264 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testAddReplicasForAllShards} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#1254035363 07:08:12,264 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard cars 07:08:12,264 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-people-oper_testAddReplicasForAllShards 07:08:12,264 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars, people] 07:08:12,265 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testAddReplicasForAllShards, leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, leaderPayloadVersion=13] 07:08:12,265 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-config_testAddReplicasForAllShards 07:08:12,265 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [cars] 07:08:12,265 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-pets-config_testAddReplicasForAllShards} for shard pets from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#1993622414 07:08:12,265 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard pets 07:08:12,265 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-pets-config_testAddReplicasForAllShards 07:08:12,265 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [pets, cars] 07:08:12,265 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testAddReplicasForAllShards, leaderId=member-1-shard-people-config_testAddReplicasForAllShards, leaderPayloadVersion=13] 07:08:12,266 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 07:08:12,266 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 07:08:12,266 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 07:08:12,266 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 07:08:12,266 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]] 07:08:12,266 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 07:08:12,266 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 07:08:12,266 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Re-initializing to Follower with previous leaderId member-1-shard-cars-config_testAddReplicasForAllShards 07:08:12,266 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 2 07:08:12,266 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - onSuccess for shard pets, type Config 07:08:12,266 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 3 07:08:12,267 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 07:08:12,267 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddReplicasForAllShards status sync done true 07:08:12,267 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:12,267 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-people-config_testAddReplicasForAllShards} for shard people from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#2124651423 07:08:12,267 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard people 07:08:12,267 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]] 07:08:12,267 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]] 07:08:12,267 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]] 07:08:12,267 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 07:08:12,267 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Re-initializing to Follower with previous leaderId member-1-shard-people-oper_testAddReplicasForAllShards 07:08:12,267 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-people-config_testAddReplicasForAllShards 07:08:12,267 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 07:08:12,267 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [pets, cars, people] 07:08:12,267 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:12,267 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 07:08:12,267 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Re-initializing to Follower with previous leaderId member-1-shard-cars-oper_testAddReplicasForAllShards 07:08:12,267 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 07:08:12,267 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testAddReplicasForAllShards status sync done true 07:08:12,267 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Re-initializing to Follower with previous leaderId member-1-shard-people-config_testAddReplicasForAllShards 07:08:12,267 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 07:08:12,267 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 07:08:12,267 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 07:08:12,268 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:12,268 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 4 07:08:12,268 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 07:08:12,268 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 5 07:08:12,268 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 07:08:12,268 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 6 07:08:12,268 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 07:08:12,268 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testAddReplicasForAllShards, leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, leaderPayloadVersion=13] 07:08:12,268 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testAddReplicasForAllShards status sync done true 07:08:12,268 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddReplicasForAllShards status sync done true 07:08:12,268 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Successfully deleted prior snapshots 07:08:12,268 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Successfully deleted prior snapshots 07:08:12,268 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Successfully deleted prior snapshots 07:08:12,268 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Successfully deleted prior snapshots 07:08:12,268 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 07:08:12,268 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testAddReplicasForAllShards updated: matchIndex: 0, nextIndex: 1 07:08:12,268 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testAddReplicasForAllShards updated: matchIndex: 0, nextIndex: 1 07:08:12,269 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 07:08:12,269 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 07:08:12,269 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Setting last applied to 0 07:08:12,269 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Moving last applied index from -1 to 0 07:08:12,269 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-pets-config_testAddReplicasForAllShards, address=null, votingState=VOTING]] 07:08:12,269 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]] 07:08:12,268 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:12,269 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Log size is now 1 07:08:12,269 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=member-2-shard-pets-config_testAddReplicasForAllShards, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:12,269 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:12,269 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-pets-config_testAddReplicasForAllShards, leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, leaderPayloadVersion=13] 07:08:12,269 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Setting last applied to 0 07:08:12,269 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 07:08:12,269 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Moving last applied index from -1 to 0 07:08:12,269 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]] 07:08:12,269 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 07:08:12,269 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 07:08:12,270 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:12,270 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testAddReplicasForAllShards updated: matchIndex: 0, nextIndex: 1 07:08:12,270 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 07:08:12,270 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Setting last applied to 0 07:08:12,270 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Moving last applied index from -1 to 0 07:08:12,270 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]] 07:08:12,270 AM [cluster-test-shard-dispatcher-32] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 07:08:12,270 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-1074180417] 07:08:12,270 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 07:08:12,270 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:12,270 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 07:08:12,270 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testAddReplicasForAllShards updated: matchIndex: 0, nextIndex: 1 07:08:12,270 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 07:08:12,270 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 07:08:12,270 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Setting last applied to 0 07:08:12,270 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 07:08:12,270 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Moving last applied index from -1 to 0 07:08:12,270 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]] 07:08:12,271 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Successfully deleted prior snapshots 07:08:12,271 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:12,271 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]] 07:08:12,271 AM [cluster-test-shard-dispatcher-32] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 07:08:12,271 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 07:08:12,271 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Re-initializing to Follower with previous leaderId member-1-shard-pets-config_testAddReplicasForAllShards 07:08:12,271 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 07:08:12,271 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 07:08:12,271 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:12,271 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testAddReplicasForAllShards status sync done true 07:08:12,274 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards#-290509940] 07:08:12,274 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards#-290509940] 07:08:12,275 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-1074180417] 07:08:12,275 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-pets-config_testAddReplicasForAllShards updated: matchIndex: 0, nextIndex: 1 07:08:12,275 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 07:08:12,275 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Setting last applied to 0 07:08:12,275 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Moving last applied index from -1 to 0 07:08:12,275 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]] 07:08:12,276 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:12,276 AM [cluster-test-shard-dispatcher-32] [INFO] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 07:08:12,276 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 07:08:12,276 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 07:08:12,276 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards#1085996316] 07:08:12,276 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards#1085996316] 07:08:12,276 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-1074180417] 07:08:12,277 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : pets 07:08:12,277 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards#-810230011] 07:08:12,277 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards#-810230011] 07:08:12,277 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#2123957484] 07:08:12,277 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 07:08:12,277 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards#-242761008] 07:08:12,277 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards#-242761008] 07:08:12,277 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#2123957484] 07:08:12,277 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 07:08:12,278 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards#-277285415] 07:08:12,278 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards#-277285415] 07:08:12,279 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Commit index set to 0 07:08:12,279 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 07:08:12,279 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Setting last applied to 0 07:08:12,279 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Moving last applied index from -1 to 0 07:08:12,279 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]] 07:08:12,280 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 07:08:12,280 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testAddReplicasForAllShards status sync done false 07:08:12,280 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Commit index set to 0 07:08:12,280 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 07:08:12,280 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Setting last applied to 0 07:08:12,280 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Moving last applied index from -1 to 0 07:08:12,280 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]] 07:08:12,280 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 07:08:12,280 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddReplicasForAllShards status sync done false 07:08:12,280 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Commit index set to 0 07:08:12,280 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 07:08:12,280 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Setting last applied to 0 07:08:12,280 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Moving last applied index from -1 to 0 07:08:12,280 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]] 07:08:12,280 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Commit index set to 0 07:08:12,281 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 07:08:12,281 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Commit index set to 0 07:08:12,281 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 07:08:12,281 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Setting last applied to 0 07:08:12,281 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 07:08:12,281 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Moving last applied index from -1 to 0 07:08:12,281 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Setting last applied to 0 07:08:12,281 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Moving last applied index from -1 to 0 07:08:12,281 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]] 07:08:12,281 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]] 07:08:12,281 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 07:08:12,281 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 07:08:12,281 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testAddReplicasForAllShards status sync done false 07:08:12,281 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testAddReplicasForAllShards status sync done false 07:08:12,281 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddReplicasForAllShards status sync done false 07:08:12,282 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 07:08:12,283 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-pets-config_testAddReplicasForAllShards 07:08:12,283 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testAddReplicasForAllShards 07:08:12,283 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testAddReplicasForAllShards 07:08:12,283 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 3 shards 07:08:12,283 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: onShutDown 07:08:12,283 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: onShutDown 07:08:12,283 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: onShutDown 07:08:12,284 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Stopping Shard member-2-shard-pets-config_testAddReplicasForAllShards 07:08:12,284 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-2-shard-people-config_testAddReplicasForAllShards 07:08:12,284 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testAddReplicasForAllShards 07:08:12,285 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 07:08:12,285 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 07:08:12,286 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 07:08:12,286 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testAddReplicasForAllShards 07:08:12,286 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testAddReplicasForAllShards 07:08:12,286 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-no-leader-oper_testAddReplicasForAllShards 07:08:12,287 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 3 shards 07:08:12,287 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: onShutDown 07:08:12,287 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: onShutDown 07:08:12,287 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: onShutDown 07:08:12,287 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testAddReplicasForAllShards 07:08:12,288 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Stopping Shard member-2-shard-no-leader-oper_testAddReplicasForAllShards 07:08:12,288 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testAddReplicasForAllShards 07:08:12,288 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 07:08:12,288 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 07:08:12,290 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:08:12,290 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:08:12,315 AM [cluster-test-pekko.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:08:12,315 AM [cluster-test-pekko.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:08:12,318 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 07:08:12,346 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 07:08:12,347 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-pets-config_testAddReplicasForAllShards 07:08:12,347 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testAddReplicasForAllShards 07:08:12,347 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testAddReplicasForAllShards 07:08:12,347 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 3 shards 07:08:12,348 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: onShutDown 07:08:12,349 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Initiating leader transfer 07:08:12,350 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-pets-config_testAddReplicasForAllShards, leaderId=null, leaderPayloadVersion=13] 07:08:12,351 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: onShutDown 07:08:12,351 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Initiating leader transfer 07:08:12,351 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: onShutDown 07:08:12,351 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Initiating leader transfer 07:08:12,352 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testAddReplicasForAllShards, leaderId=null, leaderPayloadVersion=13] 07:08:12,352 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testAddReplicasForAllShards, leaderId=null, leaderPayloadVersion=13] 07:08:12,353 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5cf9aa78 07:08:12,353 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddReplicasForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5cf9aa78 07:08:12,353 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@25c4f184 07:08:12,353 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddReplicasForAllShards: pauseLeader successfully completed - doing transfer 07:08:12,353 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-config_testAddReplicasForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@25c4f184 07:08:12,353 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Attempting to transfer leadership 07:08:12,353 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testAddReplicasForAllShards: pauseLeader successfully completed - doing transfer 07:08:12,353 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Attempting to transfer leadership 07:08:12,353 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:12,354 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:12,354 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@29a1429d 07:08:12,354 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-pets-config_testAddReplicasForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@29a1429d 07:08:12,354 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testAddReplicasForAllShards: pauseLeader successfully completed - doing transfer 07:08:12,354 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Attempting to transfer leadership 07:08:12,354 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:13,317 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [5178423692212300979] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 07:08:18,122 AM [cluster-test-pekko.actor.default-dispatcher-22] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2559, Up)]. 07:08:18,124 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:18,124 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:18,503 AM [cluster-test-pekko.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 07:08:18,819 AM [cluster-test-pekko.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 07:08:19,180 AM [cluster-test-pekko.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 07:08:19,458 AM [cluster-test-pekko.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 07:08:19,778 AM [cluster-test-pekko.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 07:08:20,93 AM [cluster-test-pekko.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 07:08:20,197 AM [cluster-test-pekko.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 07:08:20,737 AM [cluster-test-pekko.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 07:08:21,59 AM [cluster-test-pekko.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 07:08:21,218 AM [cluster-test-pekko.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 07:08:21,366 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddReplicasForAllShards: leader transfer aborted 07:08:21,366 AM [cluster-test-shard-dispatcher-33] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddReplicasForAllShards: Failed to transfer leadership in 9.015 s 07:08:21,366 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: In unpauseLeader 07:08:21,366 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: leader transfer failed - sending PoisonPill 07:08:21,366 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Leadership transfer expired 07:08:21,366 AM [cluster-test-shard-dispatcher-16] [WARN] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: At least 1 followers need to be active, Switching member-1-shard-people-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 07:08:21,367 AM [cluster-test-shard-dispatcher-33] [WARN] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: At least 1 followers need to be active, Switching member-1-shard-cars-config_testAddReplicasForAllShards from Leader to IsolatedLeader 07:08:21,367 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (IsolatedLeader): Election: Leader has following peers: [member-2-shard-people-oper_testAddReplicasForAllShards] 07:08:21,367 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (IsolatedLeader): Election: Leader has following peers: [member-2-shard-cars-config_testAddReplicasForAllShards] 07:08:21,367 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testAddReplicasForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:21,368 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddReplicasForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:21,368 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 07:08:21,368 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 07:08:21,368 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 07:08:21,368 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 07:08:21,368 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 07:08:21,368 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 07:08:21,368 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddReplicasForAllShards from Leader to IsolatedLeader 07:08:21,369 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 07:08:21,369 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testAddReplicasForAllShards 07:08:21,375 AM [cluster-test-shard-dispatcher-32] [WARN] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: At least 1 followers need to be active, Switching member-1-shard-cars-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 07:08:21,376 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (IsolatedLeader): Election: Leader has following peers: [member-2-shard-cars-oper_testAddReplicasForAllShards] 07:08:21,376 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddReplicasForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:21,376 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 07:08:21,376 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 07:08:21,376 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 07:08:21,376 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 07:08:21,385 AM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testAddReplicasForAllShards: leader transfer aborted 07:08:21,385 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testAddReplicasForAllShards: leader transfer aborted 07:08:21,385 AM [cluster-test-shard-dispatcher-33] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testAddReplicasForAllShards: Failed to transfer leadership in 9.034 s 07:08:21,385 AM [cluster-test-shard-dispatcher-32] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testAddReplicasForAllShards: Failed to transfer leadership in 9.035 s 07:08:21,385 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: In unpauseLeader 07:08:21,385 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: In unpauseLeader 07:08:21,385 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: leader transfer failed - sending PoisonPill 07:08:21,385 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Leadership transfer expired 07:08:21,385 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: leader transfer failed - sending PoisonPill 07:08:21,385 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Leadership transfer expired 07:08:21,386 AM [cluster-test-shard-dispatcher-33] [WARN] Shard - member-1-shard-people-config_testAddReplicasForAllShards: At least 1 followers need to be active, Switching member-1-shard-people-config_testAddReplicasForAllShards from Leader to IsolatedLeader 07:08:21,386 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (IsolatedLeader): Election: Leader has following peers: [member-2-shard-people-config_testAddReplicasForAllShards] 07:08:21,386 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testAddReplicasForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:21,386 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-1-shard-pets-config_testAddReplicasForAllShards 07:08:21,387 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 07:08:21,387 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 07:08:21,387 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 07:08:21,387 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testAddReplicasForAllShards from Leader to IsolatedLeader 07:08:21,387 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-1-shard-people-config_testAddReplicasForAllShards 07:08:21,388 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 07:08:21,388 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 07:08:21,389 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 07:08:21,390 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testAddReplicasForAllShards 07:08:21,391 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testAddReplicasForAllShards 07:08:21,391 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 07:08:21,391 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: onShutDown 07:08:21,391 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: onShutDown 07:08:21,392 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testAddReplicasForAllShards 07:08:21,392 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testAddReplicasForAllShards 07:08:21,393 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 07:08:21,393 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 07:08:21,395 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:08:21,395 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:08:21,400 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [01/31/2025 07:08:21.404] [cluster-test-pekko.actor.internal-dispatcher-9] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:08:21,438 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:08:21,454 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [1212651693236138869] 07:08:21,456 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 07:08:21,461 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 07:08:21,461 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 07:08:21,462 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:08:21,462 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:08:21,484 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:21,486 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 07:08:21,486 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 07:08:21,486 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 07:08:21,488 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:21,491 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStates 07:08:21,503 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 07:08:21,504 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-440670590] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:21,516 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 07:08:21,516 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testFlipMemberVotingStates 07:08:21,516 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testFlipMemberVotingStates 07:08:21,517 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:21,517 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 07:08:21,517 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 07:08:21,519 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 07:08:21,521 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@533dab84} 07:08:21,521 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} started 07:08:21,522 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:21,522 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testFlipMemberVotingStates 07:08:21,522 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:21,523 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 07:08:21,523 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 07:08:21,523 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testFlipMemberVotingStates on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates on shard member-1-shard-cars-config_testFlipMemberVotingStates whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-251673576] 07:08:21,523 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testFlipMemberVotingStates 07:08:21,523 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 07:08:21,523 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 07:08:21,523 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testFlipMemberVotingStates on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates on shard member-1-shard-people-config_testFlipMemberVotingStates whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#1813337631] 07:08:21,524 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-251673576] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:21,524 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-config_testFlipMemberVotingStates, persistent : true 07:08:21,524 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates-notifier#920373042] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:21,525 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates-notifier#920373042 created and ready for shard:member-1-shard-cars-config_testFlipMemberVotingStates 07:08:21,525 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates/shard-member-1:snapshot-read#-1954776323] 07:08:21,525 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - Shard created : member-1-shard-people-config_testFlipMemberVotingStates, persistent : true 07:08:21,525 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#1813337631] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:21,525 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates/shard-member-1:snapshot-read#-1954776323] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:21,525 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testFlipMemberVotingStates with journal batch size 1 07:08:21,526 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates-notifier#794815681] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:21,526 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates/shard-member-1:snapshot-read#-1931601689] 07:08:21,526 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates-notifier#794815681 created and ready for shard:member-1-shard-people-config_testFlipMemberVotingStates 07:08:21,526 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates/shard-member-1:snapshot-read#-1931601689] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:21,526 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:21,530 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStates 07:08:21,531 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1680659489] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:21,532 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 07:08:21,535 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - Starting recovery for member-1-shard-people-config_testFlipMemberVotingStates with journal batch size 1 07:08:21,538 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 07:08:21,538 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testFlipMemberVotingStates 07:08:21,538 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testFlipMemberVotingStates 07:08:21,539 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:21,539 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 07:08:21,539 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 07:08:21,539 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:08:21,539 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:08:21,539 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 288 07:08:21,539 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStates, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStates, address=null, votingState=VOTING]] 07:08:21,539 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:21,539 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:08:21,539 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:08:21,539 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:08:21,540 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 294 07:08:21,540 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStates, address=null, votingState=NON_VOTING]] 07:08:21,540 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:21,540 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:08:21,540 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 07:08:21,540 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:21,540 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 07:08:21,540 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testFlipMemberVotingStates is initialized 07:08:21,540 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 07:08:21,540 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testFlipMemberVotingStates is initialized 07:08:21,540 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from null to Follower 07:08:21,541 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 07:08:21,541 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:21,541 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from null to Follower 07:08:21,541 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 07:08:21,541 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 07:08:21,541 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from null to Follower 07:08:21,541 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from null to Follower 07:08:21,542 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 07:08:21,543 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@7ce8bae7} 07:08:21,543 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} started 07:08:21,543 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:21,543 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testFlipMemberVotingStates 07:08:21,544 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 07:08:21,544 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 07:08:21,544 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testFlipMemberVotingStates on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates on shard member-1-shard-cars-oper_testFlipMemberVotingStates whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#146984980] 07:08:21,544 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testFlipMemberVotingStates 07:08:21,545 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 07:08:21,545 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 07:08:21,545 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testFlipMemberVotingStates on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates on shard member-1-shard-people-oper_testFlipMemberVotingStates whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#-8331463] 07:08:21,545 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-oper_testFlipMemberVotingStates, persistent : true 07:08:21,546 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testFlipMemberVotingStates with journal batch size 1 07:08:21,546 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-people-oper_testFlipMemberVotingStates, persistent : true 07:08:21,547 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testFlipMemberVotingStates with journal batch size 1 07:08:21,548 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#146984980] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:21,548 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates-notifier#-140549629 created and ready for shard:member-1-shard-cars-oper_testFlipMemberVotingStates 07:08:21,548 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#-8331463] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:21,548 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates/shard-member-1:snapshot-read#1044381631] 07:08:21,548 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates/shard-member-1:snapshot-read#-825156790] 07:08:21,548 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates-notifier#-1006928991 created and ready for shard:member-1-shard-people-oper_testFlipMemberVotingStates 07:08:21,551 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:08:21,552 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:08:21,552 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 282 07:08:21,552 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStates, address=null, votingState=NON_VOTING]] 07:08:21,552 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:21,552 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:08:21,552 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 07:08:21,552 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:21,552 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from null to Follower 07:08:21,552 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 07:08:21,552 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testFlipMemberVotingStates is initialized 07:08:21,552 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 07:08:21,554 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from null to Follower 07:08:21,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:08:21,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:08:21,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 288 07:08:21,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStates, address=null, votingState=NON_VOTING]] 07:08:21,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:21,556 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:08:21,556 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 07:08:21,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 07:08:21,556 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testFlipMemberVotingStates is initialized 07:08:21,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:21,556 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from null to Follower 07:08:21,556 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 07:08:21,556 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from null to Follower 07:08:21,562 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:08:21,574 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-356332982335145336] 07:08:21,575 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 07:08:21,578 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:08:21,578 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 07:08:21,579 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:08:21,579 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:08:21,600 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:21,608 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:21,611 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStates 07:08:21,619 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#10783124] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:21,620 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 07:08:21,626 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 07:08:21,626 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testFlipMemberVotingStates 07:08:21,627 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testFlipMemberVotingStates 07:08:21,629 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 07:08:21,632 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStates, generation=0} started 07:08:21,632 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:21,637 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:21,639 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@56273d5d} 07:08:21,642 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStates 07:08:21,643 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:21,643 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testFlipMemberVotingStates 07:08:21,652 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#-571504725] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:21,652 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Shard created : member-2-shard-cars-config_testFlipMemberVotingStates, persistent : true 07:08:21,653 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testFlipMemberVotingStates 07:08:21,655 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates-notifier#144595640 created and ready for shard:member-2-shard-cars-config_testFlipMemberVotingStates 07:08:21,655 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates-notifier#144595640] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:21,655 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 07:08:21,655 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates/shard-member-2:snapshot-read#-95964110] 07:08:21,655 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates/shard-member-2:snapshot-read#-95964110] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:21,656 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testFlipMemberVotingStates with journal batch size 1 07:08:21,656 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Shard created : member-2-shard-people-config_testFlipMemberVotingStates, persistent : true 07:08:21,657 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 07:08:21,658 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates-notifier#2008218495 created and ready for shard:member-2-shard-people-config_testFlipMemberVotingStates 07:08:21,658 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates/shard-member-2:snapshot-read#1695942608] 07:08:21,659 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-870829185] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:21,659 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#-901496600] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:21,659 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates-notifier#2008218495] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:21,659 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates/shard-member-2:snapshot-read#1695942608] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:21,660 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} started 07:08:21,664 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:08:21,664 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:08:21,664 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 288 07:08:21,664 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStates, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStates, address=null, votingState=VOTING]] 07:08:21,664 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:21,665 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 07:08:21,671 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Starting recovery for member-2-shard-people-config_testFlipMemberVotingStates with journal batch size 1 07:08:21,664 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:08:21,672 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 07:08:21,672 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:21,672 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 07:08:21,672 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testFlipMemberVotingStates is initialized 07:08:21,672 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , received role change from null to Follower 07:08:21,672 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 07:08:21,672 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStates from null to Follower 07:08:21,672 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testFlipMemberVotingStates 07:08:21,673 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testFlipMemberVotingStates 07:08:21,673 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@22c69829} 07:08:21,674 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:21,674 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testFlipMemberVotingStates 07:08:21,675 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:08:21,675 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:08:21,675 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 294 07:08:21,675 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStates, address=null, votingState=NON_VOTING]] 07:08:21,675 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:21,675 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:08:21,675 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 07:08:21,675 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:21,676 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 07:08:21,676 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testFlipMemberVotingStates is initialized 07:08:21,676 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , received role change from null to Follower 07:08:21,676 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 07:08:21,676 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStates from null to Follower 07:08:21,677 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Shard created : member-2-shard-cars-oper_testFlipMemberVotingStates, persistent : true 07:08:21,678 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testFlipMemberVotingStates with journal batch size 1 07:08:21,678 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates/shard-member-2:snapshot-read#369448292] 07:08:21,678 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates-notifier#1408942718 created and ready for shard:member-2-shard-cars-oper_testFlipMemberVotingStates 07:08:21,679 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#1284105527] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:21,679 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates/shard-member-2:snapshot-read#369448292] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:21,680 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:08:21,680 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:08:21,680 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 282 07:08:21,680 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStates, address=null, votingState=NON_VOTING]] 07:08:21,680 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:21,680 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:08:21,680 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 07:08:21,680 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:21,680 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , received role change from null to Follower 07:08:21,681 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testFlipMemberVotingStates 07:08:21,682 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Shard created : member-2-shard-people-oper_testFlipMemberVotingStates, persistent : true 07:08:21,682 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testFlipMemberVotingStates with journal batch size 1 07:08:21,682 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates-notifier#-1007634790 created and ready for shard:member-2-shard-people-oper_testFlipMemberVotingStates 07:08:21,682 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates/shard-member-2:snapshot-read#2011386613] 07:08:21,683 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 07:08:21,683 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testFlipMemberVotingStates is initialized 07:08:21,683 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 07:08:21,684 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:08:21,684 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:08:21,684 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 288 07:08:21,684 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStates, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStates, address=null, votingState=VOTING]] 07:08:21,684 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:21,684 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:08:21,684 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 07:08:21,684 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:21,684 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , received role change from null to Follower 07:08:21,684 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStates from null to Follower 07:08:21,684 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 07:08:21,684 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testFlipMemberVotingStates is initialized 07:08:21,684 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 07:08:21,684 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStates from null to Follower 07:08:21,706 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:08:21,722 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 07:08:21,730 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:21,730 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 07:08:21,730 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 07:08:21,730 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 07:08:21,730 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 07:08:21,730 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 07:08:21,730 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-1-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 07:08:21,730 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:21,730 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 07:08:21,730 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 07:08:21,730 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 07:08:21,730 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 07:08:21,730 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 07:08:21,730 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 07:08:21,731 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 07:08:21,735 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-2555493675425138088] 07:08:21,736 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 07:08:21,739 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:08:21,739 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 07:08:21,749 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:08:21,749 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:08:21,755 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:21,757 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:21,760 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStates 07:08:21,784 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 07:08:21,790 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-788522946] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:21,798 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 07:08:21,799 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testFlipMemberVotingStates 07:08:21,799 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testFlipMemberVotingStates 07:08:21,800 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 07:08:21,805 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:21,805 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 07:08:21,805 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 07:08:21,807 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 07:08:22,71 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 07:08:22,73 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@66167a3c} 07:08:22,73 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} started 07:08:22,74 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:22,74 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testFlipMemberVotingStates 07:08:22,74 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:22,76 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 07:08:22,76 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 07:08:22,76 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-cars-config_testFlipMemberVotingStates on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates on shard member-3-shard-cars-config_testFlipMemberVotingStates whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#-147071551] 07:08:22,76 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testFlipMemberVotingStates 07:08:22,77 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:22,77 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 07:08:22,77 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 07:08:22,77 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-people-config_testFlipMemberVotingStates on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates on shard member-3-shard-people-config_testFlipMemberVotingStates whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#-1574525536] 07:08:22,78 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-3-shard-cars-config_testFlipMemberVotingStates, persistent : true 07:08:22,79 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Shard created : member-3-shard-people-config_testFlipMemberVotingStates, persistent : true 07:08:22,80 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testFlipMemberVotingStates with journal batch size 1 07:08:22,80 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#-147071551] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:22,80 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#-1574525536] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:22,80 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates/shard-member-3:snapshot-read#-1147261746] 07:08:22,80 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates-notifier#1782659304 created and ready for shard:member-3-shard-cars-config_testFlipMemberVotingStates 07:08:22,80 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates-notifier#1782659304] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:22,80 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates/shard-member-3:snapshot-read#-1147261746] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:22,81 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStates 07:08:22,81 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates-notifier#1062426717 created and ready for shard:member-3-shard-people-config_testFlipMemberVotingStates 07:08:22,81 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates-notifier#1062426717] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:22,81 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates/shard-member-3:snapshot-read#-35054068] 07:08:22,82 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates/shard-member-3:snapshot-read#-35054068] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:22,82 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:08:22,83 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-1192832717] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:22,83 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Starting recovery for member-3-shard-people-config_testFlipMemberVotingStates with journal batch size 1 07:08:22,83 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 07:08:22,84 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:08:22,84 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 288 07:08:22,84 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStates, address=null, votingState=VOTING]] 07:08:22,85 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 07:08:22,85 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:08:22,85 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testFlipMemberVotingStates 07:08:22,85 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:08:22,85 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 294 07:08:22,86 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStates, address=null, votingState=VOTING]] 07:08:22,86 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:22,86 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:08:22,86 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testFlipMemberVotingStates 07:08:22,86 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:22,87 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 07:08:22,87 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 07:08:22,87 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 07:08:22,87 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testFlipMemberVotingStates is initialized 07:08:22,87 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:22,87 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:08:22,87 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 07:08:22,87 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:22,87 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from null to Follower 07:08:22,87 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 07:08:22,88 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 07:08:22,88 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testFlipMemberVotingStates is initialized 07:08:22,88 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 07:08:22,88 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 07:08:22,89 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:22,89 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from null to Follower 07:08:22,89 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-1-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 07:08:22,89 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 07:08:22,89 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from null to Follower 07:08:22,89 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from null to Follower 07:08:22,143 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 07:08:22,145 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@4ebbb0bc} 07:08:22,145 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} started 07:08:22,146 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 07:08:22,148 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:22,148 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testFlipMemberVotingStates 07:08:22,149 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 07:08:22,149 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 07:08:22,149 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testFlipMemberVotingStates on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates on shard member-3-shard-cars-oper_testFlipMemberVotingStates whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#-1586243575] 07:08:22,149 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testFlipMemberVotingStates 07:08:22,150 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 07:08:22,150 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 07:08:22,150 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testFlipMemberVotingStates on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates on shard member-3-shard-people-oper_testFlipMemberVotingStates whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#-330172143] 07:08:22,151 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#-330172143] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:22,151 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Shard created : member-3-shard-people-oper_testFlipMemberVotingStates, persistent : true 07:08:22,152 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates-notifier#-243529961 created and ready for shard:member-3-shard-people-oper_testFlipMemberVotingStates 07:08:22,152 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates-notifier#-243529961] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:22,152 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates/shard-member-3:snapshot-read#-337342272] 07:08:22,153 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testFlipMemberVotingStates with journal batch size 1 07:08:22,153 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Shard created : member-3-shard-cars-oper_testFlipMemberVotingStates, persistent : true 07:08:22,154 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates-notifier#1469685495 created and ready for shard:member-3-shard-cars-oper_testFlipMemberVotingStates 07:08:22,154 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates/shard-member-3:snapshot-read#-1480999522] 07:08:22,158 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:08:22,158 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:08:22,158 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 288 07:08:22,158 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStates, address=null, votingState=VOTING]] 07:08:22,158 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:22,158 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:08:22,158 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 07:08:22,158 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testFlipMemberVotingStates is initialized 07:08:22,158 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 07:08:22,158 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:22,158 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from null to Follower 07:08:22,158 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 07:08:22,159 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 07:08:22,159 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testFlipMemberVotingStates with journal batch size 1 07:08:22,159 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from null to Follower 07:08:22,160 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:08:22,160 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:08:22,160 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 282 07:08:22,160 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStates, address=null, votingState=VOTING]] 07:08:22,160 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:22,160 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:08:22,160 AM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 07:08:22,160 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:22,160 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from null to Follower 07:08:22,160 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 07:08:22,160 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testFlipMemberVotingStates is initialized 07:08:22,160 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 07:08:22,160 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 07:08:22,160 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from null to Follower 07:08:22,483 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 07:08:22,484 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 07:08:22,486 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:08:22,486 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:08:22,486 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 07:08:22,486 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 07:08:22,486 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 07:08:22,486 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 07:08:22,486 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 07:08:22,486 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 07:08:22,486 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 07:08:22,486 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 07:08:22,486 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 07:08:22,486 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:22,486 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 07:08:22,486 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 07:08:22,486 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 07:08:22,486 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 07:08:22,486 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 07:08:22,486 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:22,487 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 07:08:22,487 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 07:08:22,487 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 07:08:22,487 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 07:08:22,487 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 07:08:22,487 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 07:08:22,487 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 07:08:22,487 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - Peer address for peer member-2-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 07:08:22,491 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 07:08:22,491 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 07:08:22,508 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:08:22,508 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 07:08:22,508 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 07:08:22,508 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 07:08:22,508 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 07:08:22,508 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 07:08:22,508 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-3-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 07:08:22,509 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:08:22,509 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 07:08:22,509 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 07:08:22,509 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 07:08:22,509 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 07:08:22,510 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 07:08:22,510 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 07:08:22,510 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:22,510 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 07:08:22,510 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 07:08:22,510 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 07:08:22,510 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 07:08:22,510 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:22,510 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 07:08:22,510 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 07:08:22,510 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 07:08:22,510 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 07:08:22,572 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Up), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Up)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 07:08:22,572 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Received ElectionTimeout - switching to Candidate 07:08:22,572 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testFlipMemberVotingStates] 07:08:22,572 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 2 07:08:22,573 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Sending RequestVote{term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1} to peer member-2-shard-cars-config_testFlipMemberVotingStates 07:08:22,573 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 07:08:22,574 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 07:08:22,574 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:22,574 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 07:08:22,574 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 07:08:22,581 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:08:22,582 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Cluster state: [] 07:08:22,582 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1} with higher term appears reachable, updating term. 07:08:22,582 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 07:08:22,582 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): In requestVote: RequestVote{term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1} - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 07:08:22,582 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 07:08:22,587 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply{term=2, voteGranted=true}, my term is 2 07:08:22,587 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply{term=2, voteGranted=true}, current voteCount: 1 07:08:22,587 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 07:08:22,588 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Election: Leader has following peers: [member-3-shard-cars-config_testFlipMemberVotingStates, member-2-shard-cars-config_testFlipMemberVotingStates] 07:08:22,588 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:22,589 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:22,591 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries{term=2, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 07:08:22,592 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:22,592 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStates status sync done false 07:08:22,593 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done false 07:08:22,593 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:22,596 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 07:08:22,596 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 07:08:22,596 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 07:08:22,596 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:22,596 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 07:08:22,596 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to PreLeader 07:08:22,596 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Candidate to PreLeader 07:08:22,596 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:22,596 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 07:08:22,596 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:08:22,596 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:22,596 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:08:22,597 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:08:22,597 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 07:08:22,597 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:08:22,597 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStates 07:08:22,597 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-3-shard-cars-config_testFlipMemberVotingStates: AppendEntries{term=2, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:08:22,599 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:08:22,599 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStates 07:08:22,601 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Up), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Up)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 07:08:22,602 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Received ElectionTimeout - switching to Candidate 07:08:22,602 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-oper_testFlipMemberVotingStates] 07:08:22,602 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 2 07:08:22,602 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStates: AppendEntries{term=2, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:08:22,602 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Sending RequestVote{term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1} to peer member-2-shard-cars-oper_testFlipMemberVotingStates 07:08:22,602 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 07:08:22,602 AM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 07:08:22,602 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:22,602 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 07:08:22,602 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 07:08:22,605 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:08:22,605 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 07:08:22,605 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 07:08:22,605 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:08:22,605 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:08:22,605 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 2 07:08:22,605 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:22,605 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:08:22,605 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Cluster state: [] 07:08:22,605 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1} with higher term appears reachable, updating term. 07:08:22,605 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 07:08:22,606 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): In requestVote: RequestVote{term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1} - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 07:08:22,606 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 07:08:22,606 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:08:22,606 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:08:22,606 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:08:22,607 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 07:08:22,607 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 07:08:22,607 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:08:22,607 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:08:22,607 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 2 07:08:22,607 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=member-3-shard-cars-config_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:22,601 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Up), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Up)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 07:08:22,607 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Received ElectionTimeout - switching to Candidate 07:08:22,607 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done true 07:08:22,607 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-config_testFlipMemberVotingStates] 07:08:22,607 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 2 07:08:22,607 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Sending RequestVote{term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1} to peer member-2-shard-people-config_testFlipMemberVotingStates 07:08:22,607 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 07:08:22,607 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 07:08:22,607 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:08:22,608 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:08:22,608 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStates status sync done true 07:08:22,607 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:22,608 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 07:08:22,608 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 07:08:22,609 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 07:08:22,609 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:08:22,609 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:08:22,613 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:08:22,613 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Cluster state: [] 07:08:22,613 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1} with higher term appears reachable, updating term. 07:08:22,613 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 07:08:22,613 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): In requestVote: RequestVote{term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1} - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 07:08:22,613 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 07:08:22,614 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 07:08:22,614 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:08:22,614 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 07:08:22,614 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Setting last applied to 0 07:08:22,614 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from -1 to 0 07:08:22,614 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply{term=2, voteGranted=true}, my term is 2 07:08:22,614 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 07:08:22,614 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply{term=2, voteGranted=true}, current voteCount: 1 07:08:22,614 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Setting last applied to 1 07:08:22,614 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 0 to 1 07:08:22,614 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:08:22,614 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 07:08:22,614 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:22,614 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 07:08:22,615 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Election: Leader has following peers: [member-2-shard-cars-oper_testFlipMemberVotingStates, member-3-shard-cars-oper_testFlipMemberVotingStates] 07:08:22,615 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:22,615 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:22,615 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]] - lastApplied: 1, lastIndex: 1 07:08:22,615 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 07:08:22,615 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 07:08:22,615 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-3-shard-cars-config_testFlipMemberVotingStates, member-2-shard-cars-config_testFlipMemberVotingStates] 07:08:22,615 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 07:08:22,615 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to PreLeader 07:08:22,615 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:22,615 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:22,615 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 07:08:22,615 AM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 07:08:22,616 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from PreLeader to Leader 07:08:22,615 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:08:22,616 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 07:08:22,616 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:08:22,616 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 07:08:22,616 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:22,616 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:08:22,616 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Candidate to PreLeader 07:08:22,616 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from PreLeader to Leader 07:08:22,616 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:08:22,617 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:08:22,622 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply{term=2, voteGranted=true}, my term is 2 07:08:22,623 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply{term=2, voteGranted=true}, current voteCount: 1 07:08:22,623 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 07:08:22,623 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Election: Leader has following peers: [member-2-shard-people-config_testFlipMemberVotingStates, member-3-shard-people-config_testFlipMemberVotingStates] 07:08:22,623 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:22,623 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:22,623 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 07:08:22,623 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 07:08:22,623 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 07:08:22,623 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to PreLeader 07:08:22,623 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:22,623 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:08:22,623 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:08:22,623 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Candidate to PreLeader 07:08:22,623 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 07:08:22,624 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:08:22,624 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:08:22,625 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:08:22,626 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries{term=2, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 07:08:22,626 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:22,627 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStates status sync done false 07:08:22,627 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Commit index set to 1 07:08:22,627 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 07:08:22,627 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 0 07:08:22,627 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from -1 to 0 07:08:22,627 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 07:08:22,627 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 1 07:08:22,627 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 0 to 1 07:08:22,627 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:08:22,627 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 07:08:22,628 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStates status sync done true 07:08:22,629 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries{term=2, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 07:08:22,629 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStates, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:22,629 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStates is ready 07:08:22,629 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStates status sync done false 07:08:22,630 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStates status sync done true 07:08:22,630 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 07:08:22,630 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:08:22,630 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStates 07:08:22,630 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-3-shard-cars-oper_testFlipMemberVotingStates: AppendEntries{term=2, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:08:22,630 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStates 07:08:22,632 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:08:22,632 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 07:08:22,632 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 07:08:22,632 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:08:22,632 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:08:22,632 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 2 07:08:22,632 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=member-3-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:22,632 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:08:22,633 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 07:08:22,633 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:08:22,633 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStates 07:08:22,633 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-3-shard-people-config_testFlipMemberVotingStates: AppendEntries{term=2, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:08:22,634 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:08:22,635 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 07:08:22,635 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 07:08:22,635 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:08:22,635 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:08:22,635 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 2 07:08:22,635 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=member-3-shard-people-config_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:22,636 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:08:22,636 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:08:22,636 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:08:22,636 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 07:08:22,636 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 07:08:22,636 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 07:08:22,636 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 07:08:22,636 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 07:08:22,636 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 07:08:22,636 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 07:08:22,636 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 07:08:22,636 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStates is ready 07:08:22,636 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:22,636 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 07:08:22,636 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:22,636 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 07:08:22,636 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 07:08:22,636 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 07:08:22,636 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 07:08:22,636 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 07:08:22,636 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 07:08:22,636 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 07:08:22,636 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 07:08:22,636 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-2-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 07:08:22,636 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 07:08:22,636 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 07:08:22,637 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 07:08:22,637 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStates 07:08:22,637 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:08:22,637 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Commit index set to 1 07:08:22,637 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 07:08:22,637 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 0 07:08:22,637 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from -1 to 0 07:08:22,637 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 07:08:22,637 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 1 07:08:22,637 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 0 to 1 07:08:22,637 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:08:22,637 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 07:08:22,637 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 07:08:22,637 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done false 07:08:22,638 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:22,638 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done true 07:08:22,638 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStates, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:22,639 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 07:08:22,639 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done false 07:08:22,640 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done true 07:08:22,640 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 07:08:22,640 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:08:22,642 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Up), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Up)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 07:08:22,642 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Received ElectionTimeout - switching to Candidate 07:08:22,642 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStates is ready 07:08:22,642 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-oper_testFlipMemberVotingStates] 07:08:22,642 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 2 07:08:22,642 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Sending RequestVote{term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1} to peer member-2-shard-people-oper_testFlipMemberVotingStates 07:08:22,642 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 07:08:22,642 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 07:08:22,642 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:22,642 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 07:08:22,643 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 07:08:22,646 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:08:22,646 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Cluster state: [] 07:08:22,646 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1} with higher term appears reachable, updating term. 07:08:22,646 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 07:08:22,646 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): In requestVote: RequestVote{term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1} - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 07:08:22,646 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 07:08:22,647 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 07:08:22,647 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:08:22,647 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:08:22,647 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStates 07:08:22,647 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStates: AppendEntries{term=2, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:08:22,647 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 07:08:22,647 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:08:22,647 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:08:22,647 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStates 07:08:22,647 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:08:22,647 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStates 07:08:22,647 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStates: AppendEntries{term=2, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:08:22,648 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:08:22,648 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStates 07:08:22,648 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply{term=2, voteGranted=true}, my term is 2 07:08:22,648 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply{term=2, voteGranted=true}, current voteCount: 1 07:08:22,648 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 07:08:22,648 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Election: Leader has following peers: [member-2-shard-people-oper_testFlipMemberVotingStates, member-3-shard-people-oper_testFlipMemberVotingStates] 07:08:22,648 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:22,649 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:22,649 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 07:08:22,649 AM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 07:08:22,649 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 07:08:22,649 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to PreLeader 07:08:22,649 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:08:22,649 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:08:22,649 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 07:08:22,649 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:22,649 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Candidate to PreLeader 07:08:22,649 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:08:22,649 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:08:22,650 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries{term=2, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 07:08:22,650 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:08:22,650 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 07:08:22,650 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:22,650 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 07:08:22,650 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:08:22,650 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:08:22,650 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 07:08:22,651 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 2 07:08:22,651 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:08:22,651 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 07:08:22,651 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 07:08:22,651 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:08:22,651 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:08:22,651 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:22,651 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 2 07:08:22,651 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:22,651 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:08:22,651 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:08:22,651 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:08:22,651 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStates status sync done false 07:08:22,651 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done false 07:08:22,651 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:22,651 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 07:08:22,652 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done true 07:08:22,653 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 07:08:22,653 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:08:22,653 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStates status sync done true 07:08:22,653 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStates 07:08:22,654 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-3-shard-people-oper_testFlipMemberVotingStates: AppendEntries{term=2, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:08:22,654 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 07:08:22,654 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:08:22,654 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 07:08:22,654 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Setting last applied to 0 07:08:22,654 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Moving last applied index from -1 to 0 07:08:22,654 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 07:08:22,654 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Setting last applied to 1 07:08:22,654 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 0 to 1 07:08:22,654 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:08:22,654 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 07:08:22,654 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:22,654 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]] - lastApplied: 1, lastIndex: 1 07:08:22,654 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-people-config_testFlipMemberVotingStates, member-3-shard-people-config_testFlipMemberVotingStates] 07:08:22,654 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:22,654 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:22,654 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 07:08:22,654 AM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 07:08:22,655 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 07:08:22,655 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from PreLeader to Leader 07:08:22,655 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from PreLeader to Leader 07:08:22,655 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 07:08:22,655 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:08:22,655 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 07:08:22,655 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Setting last applied to 0 07:08:22,655 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from -1 to 0 07:08:22,655 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 07:08:22,655 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Setting last applied to 1 07:08:22,655 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 0 to 1 07:08:22,655 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:08:22,655 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 07:08:22,655 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:22,655 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 07:08:22,655 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:08:22,655 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]] - lastApplied: 1, lastIndex: 1 07:08:22,655 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:08:22,655 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStates 07:08:22,655 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:08:22,655 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 07:08:22,655 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStates: AppendEntries{term=2, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:08:22,655 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-cars-oper_testFlipMemberVotingStates, member-3-shard-cars-oper_testFlipMemberVotingStates] 07:08:22,655 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} is now ready 07:08:22,655 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 07:08:22,655 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 07:08:22,655 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 07:08:22,655 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:08:22,655 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:08:22,655 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:22,655 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 2 07:08:22,655 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=member-3-shard-people-oper_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:22,655 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:22,656 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Commit index set to 1 07:08:22,656 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 07:08:22,656 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 0 07:08:22,656 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:08:22,656 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Moving last applied index from -1 to 0 07:08:22,656 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 07:08:22,656 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStates 07:08:22,656 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 1 07:08:22,656 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 0 to 1 07:08:22,656 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:08:22,657 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 07:08:22,657 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Commit index set to 1 07:08:22,657 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 07:08:22,657 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 0 07:08:22,657 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Moving last applied index from -1 to 0 07:08:22,657 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 07:08:22,657 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 1 07:08:22,657 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 0 to 1 07:08:22,657 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:08:22,657 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 07:08:22,657 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 07:08:22,657 AM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 07:08:22,657 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from PreLeader to Leader 07:08:22,657 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 07:08:22,657 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from PreLeader to Leader 07:08:22,656 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:08:22,657 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 07:08:22,657 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStates 07:08:22,658 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:08:22,659 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Commit index set to 1 07:08:22,659 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 07:08:22,659 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 0 07:08:22,659 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from -1 to 0 07:08:22,659 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 07:08:22,659 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 1 07:08:22,659 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 0 to 1 07:08:22,659 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:08:22,659 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 07:08:22,659 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 07:08:22,660 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Commit index set to 1 07:08:22,660 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 07:08:22,660 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 07:08:22,660 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 0 07:08:22,660 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:08:22,660 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from -1 to 0 07:08:22,660 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 07:08:22,660 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 1 07:08:22,660 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 0 to 1 07:08:22,660 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:08:22,660 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:08:22,660 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 07:08:22,661 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:08:22,661 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 07:08:22,661 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 07:08:22,661 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:08:22,661 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:08:22,661 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 2 07:08:22,661 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:22,661 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:08:22,663 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 07:08:22,664 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:08:22,664 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 07:08:22,664 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Setting last applied to 0 07:08:22,664 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from -1 to 0 07:08:22,664 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 07:08:22,664 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Setting last applied to 1 07:08:22,664 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 0 to 1 07:08:22,664 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:08:22,664 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 07:08:22,664 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:22,664 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]] - lastApplied: 1, lastIndex: 1 07:08:22,664 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-3-shard-people-oper_testFlipMemberVotingStates, member-2-shard-people-oper_testFlipMemberVotingStates] 07:08:22,664 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:22,664 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:22,664 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 07:08:22,664 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 07:08:22,664 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 07:08:22,664 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from PreLeader to Leader 07:08:22,665 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from PreLeader to Leader 07:08:22,665 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 07:08:22,665 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} is now ready 07:08:22,665 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 07:08:22,665 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} is now ready 07:08:22,665 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 07:08:22,665 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} is now ready 07:08:22,665 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-440670590] 07:08:22,665 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:08:22,666 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-251673576] 07:08:22,666 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-251673576] 07:08:22,667 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Commit index set to 1 07:08:22,667 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 07:08:22,667 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 0 07:08:22,667 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from -1 to 0 07:08:22,667 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 07:08:22,667 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 1 07:08:22,667 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 0 to 1 07:08:22,667 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:08:22,667 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 07:08:22,669 AM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 07:08:22,669 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStates 07:08:22,669 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStates 07:08:22,669 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 07:08:22,669 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 07:08:22,669 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Commit index set to 1 07:08:22,669 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 07:08:22,669 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 0 07:08:22,669 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from -1 to 0 07:08:22,669 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 07:08:22,669 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 1 07:08:22,669 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 0 to 1 07:08:22,669 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:08:22,670 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 07:08:22,670 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 07:08:22,671 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 07:08:22,671 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 07:08:22,671 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 07:08:22,672 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 07:08:22,672 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 07:08:22,672 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 07:08:22,674 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=true, member-2-shard-cars-config_testFlipMemberVotingStates=false, member-1-shard-cars-config_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 07:08:22,674 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=true, member-1-shard-people-oper_testFlipMemberVotingStates=false, member-2-shard-people-oper_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 07:08:22,674 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=true, member-2-shard-cars-config_testFlipMemberVotingStates=false, member-1-shard-cars-config_testFlipMemberVotingStates=false}, serversVisited=[]], state: Idle 07:08:22,674 AM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=true, member-1-shard-people-oper_testFlipMemberVotingStates=false, member-2-shard-people-oper_testFlipMemberVotingStates=false}, serversVisited=[]], state: Idle 07:08:22,674 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-cars-config_testFlipMemberVotingStates peerAddress = pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 07:08:22,674 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-people-oper_testFlipMemberVotingStates peerAddress = pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 07:08:22,674 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testFlipMemberVotingStates: Not leader - forwarding to leader ActorSelection[Anchor(pekko://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates)] 07:08:22,674 AM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testFlipMemberVotingStates: Not leader - forwarding to leader ActorSelection[Anchor(pekko://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates)] 07:08:22,675 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=false, member-1-shard-cars-oper_testFlipMemberVotingStates=false, member-3-shard-cars-oper_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 07:08:22,675 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=false, member-1-shard-cars-oper_testFlipMemberVotingStates=false, member-3-shard-cars-oper_testFlipMemberVotingStates=true}, serversVisited=[]], state: Idle 07:08:22,676 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=false, member-2-shard-people-config_testFlipMemberVotingStates=false, member-3-shard-people-config_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 07:08:22,676 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=false, member-2-shard-people-config_testFlipMemberVotingStates=false, member-3-shard-people-config_testFlipMemberVotingStates=true}, serversVisited=[]], state: Idle 07:08:22,676 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-cars-oper_testFlipMemberVotingStates peerAddress = pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 07:08:22,676 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-people-config_testFlipMemberVotingStates peerAddress = pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 07:08:22,676 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testFlipMemberVotingStates: Not leader - forwarding to leader ActorSelection[Anchor(pekko://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates)] 07:08:22,676 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: Not leader - forwarding to leader ActorSelection[Anchor(pekko://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates)] 07:08:22,684 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=true, member-2-shard-cars-config_testFlipMemberVotingStates=false, member-1-shard-cars-config_testFlipMemberVotingStates=false}, serversVisited=[]], state: Idle 07:08:22,684 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-oper_testFlipMemberVotingStates=false, member-3-shard-people-oper_testFlipMemberVotingStates=true, member-1-shard-people-oper_testFlipMemberVotingStates=false}, serversVisited=[]], state: Idle 07:08:22,687 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=false, member-2-shard-people-config_testFlipMemberVotingStates=false, member-3-shard-people-config_testFlipMemberVotingStates=true}, serversVisited=[]], state: Idle 07:08:22,687 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=false, member-1-shard-cars-oper_testFlipMemberVotingStates=false, member-3-shard-cars-oper_testFlipMemberVotingStates=true}, serversVisited=[]], state: Idle 07:08:22,687 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 07:08:22,687 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 07:08:22,687 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 07:08:22,687 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 07:08:22,688 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates, votingState=VOTING]] 07:08:22,688 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates, votingState=VOTING]] 07:08:22,688 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates, votingState=NON_VOTING]] 07:08:22,688 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates, votingState=VOTING]] 07:08:22,688 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]] 07:08:22,688 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]] 07:08:22,688 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]] 07:08:22,688 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]] 07:08:22,688 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 07:08:22,688 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 07:08:22,688 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 07:08:22,688 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 07:08:22,688 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 07:08:22,688 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 07:08:22,688 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 07:08:22,688 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 07:08:22,688 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=873670aa-eec1-4207-b3d7-6b84c4f4fbd8}, logIndex: 2, isSendImmediate: true 07:08:22,688 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=ceb09e93-5856-4c8a-85e7-92f89325b009}, logIndex: 2, isSendImmediate: true 07:08:22,688 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=d7099008-6765-46a9-ab79-8a02c92ca41c}, logIndex: 2, isSendImmediate: true 07:08:22,688 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=051adf4f-f128-4beb-9a40-44b1fadfaa29}, logIndex: 2, isSendImmediate: true 07:08:22,688 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 07:08:22,688 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 07:08:22,688 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-3-shard-people-oper_testFlipMemberVotingStates 07:08:22,688 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 07:08:22,688 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-2-shard-people-config_testFlipMemberVotingStates 07:08:22,688 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStates 07:08:22,688 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 07:08:22,688 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-config_testFlipMemberVotingStates 07:08:22,688 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStates: AppendEntries{term=2, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:22,688 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-3-shard-people-oper_testFlipMemberVotingStates: AppendEntries{term=2, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:22,688 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStates: AppendEntries{term=2, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:22,688 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testFlipMemberVotingStates: AppendEntries{term=2, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:22,689 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 07:08:22,689 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 07:08:22,689 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 07:08:22,689 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStates 07:08:22,689 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-2-shard-people-oper_testFlipMemberVotingStates 07:08:22,689 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-3-shard-people-config_testFlipMemberVotingStates 07:08:22,689 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 07:08:22,689 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-config_testFlipMemberVotingStates 07:08:22,689 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testFlipMemberVotingStates: AppendEntries{term=2, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:22,689 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStates: AppendEntries{term=2, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:22,689 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-3-shard-people-config_testFlipMemberVotingStates: AppendEntries{term=2, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:22,689 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStates: AppendEntries{term=2, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:22,689 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-oper_testFlipMemberVotingStates=false, member-3-shard-people-oper_testFlipMemberVotingStates=true, member-1-shard-people-oper_testFlipMemberVotingStates=false}, serversVisited=[]] 07:08:22,689 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=false, member-2-shard-people-config_testFlipMemberVotingStates=false, member-3-shard-people-config_testFlipMemberVotingStates=true}, serversVisited=[]] 07:08:22,689 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=true, member-2-shard-cars-config_testFlipMemberVotingStates=false, member-1-shard-cars-config_testFlipMemberVotingStates=false}, serversVisited=[]] 07:08:22,689 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=false, member-1-shard-cars-oper_testFlipMemberVotingStates=false, member-3-shard-cars-oper_testFlipMemberVotingStates=true}, serversVisited=[]] 07:08:22,690 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 07:08:22,690 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 07:08:22,691 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 07:08:22,691 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 07:08:22,698 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:22,698 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 07:08:22,698 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 07:08:22,698 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 07:08:22,698 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 07:08:22,699 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates, votingState=NON_VOTING]] 07:08:22,699 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 3 07:08:22,699 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=member-3-shard-people-oper_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:22,699 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 07:08:22,699 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:22,699 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 07:08:22,699 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:22,699 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 07:08:22,699 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 07:08:22,699 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 07:08:22,699 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 07:08:22,699 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 07:08:22,699 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 07:08:22,699 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 07:08:22,699 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates, votingState=VOTING]] 07:08:22,699 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates, votingState=VOTING]] 07:08:22,699 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 2 07:08:22,699 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 2 07:08:22,699 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:22,699 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:22,699 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 07:08:22,700 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:22,700 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:22,700 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 07:08:22,700 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 07:08:22,700 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 07:08:22,700 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 07:08:22,700 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 07:08:22,700 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 07:08:22,700 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 07:08:22,700 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 07:08:22,700 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates, votingState=NON_VOTING]] 07:08:22,700 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates, votingState=NON_VOTING]] 07:08:22,700 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 2 07:08:22,700 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 2 07:08:22,700 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:22,700 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:22,700 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:22,700 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 07:08:22,700 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 07:08:22,700 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 07:08:22,700 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 07:08:22,700 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates, votingState=NON_VOTING]] 07:08:22,700 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 3 07:08:22,700 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=member-3-shard-cars-config_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:22,701 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 07:08:22,701 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 07:08:22,701 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 07:08:22,701 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 07:08:22,701 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 07:08:22,701 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 07:08:22,702 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:22,702 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 07:08:22,702 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 07:08:22,702 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 07:08:22,702 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 07:08:22,702 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates, votingState=NON_VOTING]] 07:08:22,702 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 3 07:08:22,702 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=member-3-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:22,702 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 07:08:22,703 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 07:08:22,704 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 07:08:22,704 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 07:08:22,704 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 07:08:22,704 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 07:08:22,704 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 07:08:22,704 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 07:08:22,704 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 07:08:22,704 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 07:08:22,704 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 07:08:22,704 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 07:08:22,704 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Setting last applied to 2 07:08:22,704 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 1 to 2 07:08:22,704 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 07:08:22,704 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 07:08:22,704 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:22,705 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 07:08:22,705 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 07:08:22,705 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:22,705 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 07:08:22,705 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 07:08:22,705 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 07:08:22,705 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 07:08:22,705 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates, votingState=NON_VOTING]] 07:08:22,705 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 3 07:08:22,705 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=member-3-shard-people-config_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:22,705 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 07:08:22,705 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Initiating leader transfer 07:08:22,706 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 07:08:22,706 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@74460de2 07:08:22,706 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@74460de2 07:08:22,706 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 07:08:22,706 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 07:08:22,706 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:22,706 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:22,706 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 07:08:22,706 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 07:08:22,706 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard cars 07:08:22,707 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 07:08:22,707 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 07:08:22,707 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 07:08:22,707 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard people 07:08:22,707 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 07:08:22,707 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 07:08:22,707 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 07:08:22,708 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 07:08:22,708 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 07:08:22,708 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Setting last applied to 2 07:08:22,708 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 1 to 2 07:08:22,708 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 07:08:22,708 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 07:08:22,708 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:22,708 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 07:08:22,708 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 07:08:22,708 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Initiating leader transfer 07:08:22,708 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 07:08:22,708 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3f5cf953 07:08:22,708 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3f5cf953 07:08:22,708 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 07:08:22,708 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 07:08:22,708 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 07:08:22,708 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 07:08:22,708 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 07:08:22,708 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 07:08:22,708 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Setting last applied to 2 07:08:22,708 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 1 to 2 07:08:22,708 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Setting last applied to 2 07:08:22,709 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 1 to 2 07:08:22,709 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:22,709 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 07:08:22,709 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 07:08:22,709 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 07:08:22,709 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 07:08:22,709 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:22,709 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:22,709 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:22,709 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 07:08:22,709 AM [cluster-test-shard-dispatcher-33] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 07:08:22,709 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 07:08:22,709 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 07:08:22,709 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Initiating leader transfer 07:08:22,709 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 07:08:22,709 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Initiating leader transfer 07:08:22,709 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@11ed9ff 07:08:22,709 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6a4febf6 07:08:22,709 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@11ed9ff 07:08:22,709 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6a4febf6 07:08:22,709 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 07:08:22,709 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 07:08:22,709 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 07:08:22,709 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 07:08:22,709 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:22,709 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:22,709 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:22,709 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:22,709 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 07:08:22,709 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 07:08:22,709 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 07:08:22,709 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 07:08:22,712 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 07:08:22,712 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 07:08:22,712 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 07:08:22,712 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 07:08:22,712 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 07:08:22,712 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard people 07:08:22,712 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 07:08:22,712 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 07:08:22,713 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-440670590] 07:08:22,714 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:08:22,714 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-251673576] 07:08:22,714 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-251673576] 07:08:22,714 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Commit index set to 2 07:08:22,714 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 07:08:22,714 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 2 07:08:22,714 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 1 to 2 07:08:22,714 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 07:08:22,714 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 07:08:22,714 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 07:08:22,714 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates] 07:08:22,714 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Commit index set to 2 07:08:22,714 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 07:08:22,714 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 2 07:08:22,714 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 1 to 2 07:08:22,715 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 07:08:22,715 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 07:08:22,715 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 07:08:22,715 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates] 07:08:22,715 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Commit index set to 2 07:08:22,715 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 07:08:22,715 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 2 07:08:22,715 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 1 to 2 07:08:22,715 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 07:08:22,715 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 07:08:22,715 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 07:08:22,715 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Commit index set to 2 07:08:22,715 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 07:08:22,715 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 07:08:22,715 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates] 07:08:22,715 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 07:08:22,715 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 2 07:08:22,715 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 1 to 2 07:08:22,715 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 07:08:22,715 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 07:08:22,715 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 07:08:22,715 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 07:08:22,715 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-config_testFlipMemberVotingStates] 07:08:22,715 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 07:08:22,716 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-440670590] 07:08:22,716 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 07:08:22,716 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#1813337631] 07:08:22,716 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#1813337631] 07:08:22,716 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1680659489] 07:08:22,716 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 07:08:22,716 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#146984980] 07:08:22,716 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#146984980] 07:08:22,717 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1680659489] 07:08:22,717 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 07:08:22,717 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#-8331463] 07:08:22,717 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#-8331463] 07:08:22,717 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#10783124] 07:08:22,717 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 07:08:22,718 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#-571504725] 07:08:22,718 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#-571504725] 07:08:22,718 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-oper_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: false 07:08:22,718 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#10783124] 07:08:22,718 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 07:08:22,718 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#-901496600] 07:08:22,718 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#-901496600] 07:08:22,718 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-config_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: false 07:08:22,718 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-870829185] 07:08:22,718 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 07:08:22,718 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#1284105527] 07:08:22,718 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#1284105527] 07:08:22,718 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-870829185] 07:08:22,718 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 07:08:22,718 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#1464419475] 07:08:22,719 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#1464419475] 07:08:22,719 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-788522946] 07:08:22,719 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 07:08:22,719 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#-147071551] 07:08:22,719 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#-147071551] 07:08:22,719 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-788522946] 07:08:22,719 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 07:08:22,719 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#-1574525536] 07:08:22,719 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#-1574525536] 07:08:22,720 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-1192832717] 07:08:22,720 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 07:08:22,720 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#-1586243575] 07:08:22,720 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#-1586243575] 07:08:22,720 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-1192832717] 07:08:22,720 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 07:08:22,720 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#-330172143] 07:08:22,720 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#-330172143] 07:08:22,721 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-440670590] 07:08:22,721 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:08:22,721 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-oper_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: false 07:08:22,721 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-251673576] 07:08:22,721 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-251673576] 07:08:22,722 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-config_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: false 07:08:22,724 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Commit index set to 2 07:08:22,724 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 07:08:22,725 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 2 07:08:22,725 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 1 to 2 07:08:22,725 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 07:08:22,725 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 07:08:22,725 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 07:08:22,725 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates] 07:08:22,725 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 07:08:22,725 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Commit index set to 2 07:08:22,725 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 07:08:22,725 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 2 07:08:22,725 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 1 to 2 07:08:22,725 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 07:08:22,726 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 07:08:22,726 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 07:08:22,726 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates] 07:08:22,726 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 07:08:22,727 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Commit index set to 2 07:08:22,727 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 07:08:22,727 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 2 07:08:22,727 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 1 to 2 07:08:22,727 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 07:08:22,727 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 07:08:22,727 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 07:08:22,727 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Commit index set to 2 07:08:22,727 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 07:08:22,727 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 2 07:08:22,727 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 1 to 2 07:08:22,727 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 07:08:22,728 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 07:08:22,728 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 07:08:22,728 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates] 07:08:22,728 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 07:08:22,729 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-config_testFlipMemberVotingStates] 07:08:22,729 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 07:08:22,731 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-3-shard-people-oper_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: true 07:08:22,731 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 07:08:22,731 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:22,731 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:22,731 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Leader transfer complete 07:08:22,731 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 07:08:22,736 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 07:08:22,736 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [] 07:08:22,736 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 3 07:08:22,736 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 07:08:22,736 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 07:08:22,736 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:22,736 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 07:08:22,736 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 07:08:22,736 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 07:08:22,737 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Candidate): Received ElectionTimeout 07:08:22,737 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-people-oper_testFlipMemberVotingStates, member-1-shard-people-oper_testFlipMemberVotingStates] 07:08:22,737 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:22,737 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-3-shard-cars-config_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: true 07:08:22,737 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 07:08:22,737 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:22,738 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:22,738 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 07:08:22,738 AM [cluster-test-shard-dispatcher-35] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 07:08:22,738 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 07:08:22,738 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:22,738 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-3-shard-people-config_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: true 07:08:22,738 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 07:08:22,738 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Leader transfer complete 07:08:22,738 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 07:08:22,738 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:22,738 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:22,738 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Leader transfer complete 07:08:22,738 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 07:08:22,738 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 07:08:22,738 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:22,739 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from Candidate to Leader 07:08:22,740 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 2 - updating term 07:08:22,740 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:22,741 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done false 07:08:22,741 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 07:08:22,741 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:22,742 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-3-shard-cars-oper_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: true 07:08:22,742 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 07:08:22,742 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:22,742 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:22,742 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Leader transfer complete 07:08:22,742 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 07:08:22,744 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 2 - switching to Follower 07:08:22,744 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done true 07:08:22,744 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 07:08:22,744 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 07:08:22,744 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:22,744 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Leader to Follower 07:08:22,744 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 07:08:22,744 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Leader to Follower 07:08:22,746 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 07:08:22,746 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [] 07:08:22,746 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 3 07:08:22,746 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 07:08:22,746 AM [cluster-test-shard-dispatcher-35] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 07:08:22,746 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:22,747 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 07:08:22,747 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 07:08:22,747 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 07:08:22,747 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Candidate): Received ElectionTimeout 07:08:22,747 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testFlipMemberVotingStates, member-1-shard-cars-config_testFlipMemberVotingStates] 07:08:22,747 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:22,747 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:22,747 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 07:08:22,748 AM [cluster-test-shard-dispatcher-35] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 07:08:22,748 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 07:08:22,748 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:22,748 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to Leader 07:08:22,748 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from Candidate to Leader 07:08:22,750 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 2 - updating term 07:08:22,750 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 07:08:22,750 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [] 07:08:22,750 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 07:08:22,750 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 3 07:08:22,750 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 07:08:22,750 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [] 07:08:22,750 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 07:08:22,750 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 3 07:08:22,750 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:22,750 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 07:08:22,750 AM [cluster-test-shard-dispatcher-35] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 07:08:22,750 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 07:08:22,750 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:22,750 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:22,750 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 07:08:22,750 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 07:08:22,750 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 07:08:22,750 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 07:08:22,750 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 07:08:22,751 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Candidate): Received ElectionTimeout 07:08:22,751 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-1-shard-people-config_testFlipMemberVotingStates, member-2-shard-people-config_testFlipMemberVotingStates] 07:08:22,751 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:22,751 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:22,751 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 07:08:22,751 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 07:08:22,751 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 07:08:22,751 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to Leader 07:08:22,751 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStates, leaderId=member-3-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:22,751 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Candidate): Received ElectionTimeout 07:08:22,751 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from Candidate to Leader 07:08:22,751 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStates is ready 07:08:22,751 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-cars-oper_testFlipMemberVotingStates, member-1-shard-cars-oper_testFlipMemberVotingStates] 07:08:22,751 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:22,751 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:22,752 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 07:08:22,752 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done false 07:08:22,752 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 07:08:22,752 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 07:08:22,752 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 07:08:22,752 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:22,752 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from Candidate to Leader 07:08:22,752 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 07:08:22,753 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 2 - updating term 07:08:22,754 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 2 - updating term 07:08:22,754 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStates, leaderId=member-3-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:22,754 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 07:08:22,754 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:22,754 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 07:08:22,754 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 07:08:22,754 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:22,755 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 07:08:22,756 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 2 - switching to Follower 07:08:22,756 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 07:08:22,756 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 07:08:22,756 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:22,756 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Leader to Follower 07:08:22,756 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Cannot append entries because sender's term 2 is less than 3 07:08:22,756 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Cannot append entries because sender's term 2 is less than 3 07:08:22,756 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Cannot append entries because sender's term 2 is less than 3 07:08:22,756 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done false 07:08:22,756 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done false 07:08:22,756 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 07:08:22,756 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Leader to Follower 07:08:22,757 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done true 07:08:22,758 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 07:08:22,758 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:22,758 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 07:08:22,758 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:22,760 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done true 07:08:22,760 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done true 07:08:22,760 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 2 - switching to Follower 07:08:22,760 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 07:08:22,760 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 07:08:22,760 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:22,760 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Leader to Follower 07:08:22,760 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 07:08:22,761 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Leader to Follower 07:08:22,761 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 2 - switching to Follower 07:08:22,761 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 07:08:22,761 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 07:08:22,761 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:22,761 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Leader to Follower 07:08:22,761 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 07:08:22,761 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Leader to Follower 07:08:22,772 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-251673576] 07:08:22,823 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-251673576] 07:08:22,874 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-251673576] 07:08:22,925 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-251673576] 07:08:22,975 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-251673576] 07:08:23,26 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-251673576] 07:08:23,61 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: leader changed to member-3-shard-people-oper_testFlipMemberVotingStates 07:08:23,61 AM [cluster-test-shard-dispatcher-33] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-people-oper_testFlipMemberVotingStates in 355.1 ms 07:08:23,61 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: leader transfer succeeded after change to non-voting 07:08:23,61 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-people-oper_testFlipMemberVotingStates status sync done false 07:08:23,61 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:23,63 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 07:08:23,63 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:23,65 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-people-oper_testFlipMemberVotingStates status sync done true 07:08:23,69 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: leader changed to member-3-shard-cars-config_testFlipMemberVotingStates 07:08:23,69 AM [cluster-test-shard-dispatcher-34] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-cars-config_testFlipMemberVotingStates in 360.6 ms 07:08:23,69 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: leader transfer succeeded after change to non-voting 07:08:23,69 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testFlipMemberVotingStates status sync done false 07:08:23,69 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: leader changed to member-3-shard-cars-oper_testFlipMemberVotingStates 07:08:23,69 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: leader changed to member-3-shard-people-config_testFlipMemberVotingStates 07:08:23,69 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-cars-oper_testFlipMemberVotingStates in 360.4 ms 07:08:23,69 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-people-config_testFlipMemberVotingStates in 360.4 ms 07:08:23,69 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: leader transfer succeeded after change to non-voting 07:08:23,69 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: leader transfer succeeded after change to non-voting 07:08:23,69 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-cars-oper_testFlipMemberVotingStates status sync done false 07:08:23,69 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-people-config_testFlipMemberVotingStates status sync done false 07:08:23,69 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=member-3-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:23,70 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:23,70 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 07:08:23,70 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:23,70 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 07:08:23,71 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 07:08:23,71 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:23,72 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 07:08:23,72 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:23,72 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 07:08:23,72 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:08:23,74 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testFlipMemberVotingStates status sync done true 07:08:23,74 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-people-config_testFlipMemberVotingStates status sync done true 07:08:23,74 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-cars-oper_testFlipMemberVotingStates status sync done true 07:08:23,77 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-251673576] 07:08:23,78 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1680659489] 07:08:23,78 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 07:08:23,78 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#146984980] 07:08:23,78 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#146984980] 07:08:23,78 AM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 07:08:23,78 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStates 07:08:23,79 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 07:08:23,79 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStates 07:08:23,79 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 07:08:23,79 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 07:08:23,79 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 07:08:23,79 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 07:08:23,79 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 07:08:23,80 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 07:08:23,80 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 07:08:23,80 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=true, member-2-shard-people-config_testFlipMemberVotingStates=true, member-3-shard-people-config_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 07:08:23,80 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=false, member-2-shard-cars-config_testFlipMemberVotingStates=true, member-1-shard-cars-config_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 07:08:23,80 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=true, member-2-shard-people-config_testFlipMemberVotingStates=true, member-3-shard-people-config_testFlipMemberVotingStates=false}, serversVisited=[]], state: Idle 07:08:23,80 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=false, member-2-shard-cars-config_testFlipMemberVotingStates=true, member-1-shard-cars-config_testFlipMemberVotingStates=true}, serversVisited=[]], state: Idle 07:08:23,80 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 07:08:23,80 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 07:08:23,80 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates, votingState=VOTING]] 07:08:23,80 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates, votingState=VOTING]] 07:08:23,81 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]] 07:08:23,81 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]] 07:08:23,81 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 07:08:23,81 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 07:08:23,81 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 07:08:23,81 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 07:08:23,81 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=95b92c49-6977-4aaf-85ce-f5261225b4a8}, logIndex: 3, isSendImmediate: true 07:08:23,81 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=de6d6f36-4a7b-4604-9a9c-ab24db97b4ef}, logIndex: 3, isSendImmediate: true 07:08:23,81 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 07:08:23,81 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-people-config_testFlipMemberVotingStates 07:08:23,81 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 07:08:23,81 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-config_testFlipMemberVotingStates 07:08:23,81 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-1-shard-people-config_testFlipMemberVotingStates: AppendEntries{term=3, leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:23,81 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStates: AppendEntries{term=3, leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:23,81 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 07:08:23,81 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-2-shard-people-config_testFlipMemberVotingStates 07:08:23,81 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 07:08:23,81 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-config_testFlipMemberVotingStates 07:08:23,81 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStates: AppendEntries{term=3, leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:23,81 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-1-shard-cars-config_testFlipMemberVotingStates: AppendEntries{term=3, leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:23,81 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=true, member-2-shard-people-config_testFlipMemberVotingStates=true, member-3-shard-people-config_testFlipMemberVotingStates=false}, serversVisited=[]] 07:08:23,81 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=false, member-2-shard-cars-config_testFlipMemberVotingStates=true, member-1-shard-cars-config_testFlipMemberVotingStates=true}, serversVisited=[]] 07:08:23,82 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=true, member-1-shard-cars-oper_testFlipMemberVotingStates=true, member-3-shard-cars-oper_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 07:08:23,82 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-people-config_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 07:08:23,82 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard people 07:08:23,82 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 07:08:23,82 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=true, member-1-shard-cars-oper_testFlipMemberVotingStates=true, member-3-shard-cars-oper_testFlipMemberVotingStates=false}, serversVisited=[]], state: Idle 07:08:23,82 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 07:08:23,82 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-cars-config_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 07:08:23,82 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 07:08:23,82 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 07:08:23,82 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 07:08:23,82 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 07:08:23,82 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates, votingState=VOTING]] 07:08:23,82 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]] 07:08:23,82 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=false, member-1-shard-people-oper_testFlipMemberVotingStates=true, member-2-shard-people-oper_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 07:08:23,82 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 07:08:23,82 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 07:08:23,82 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=99925643-0871-4c65-80ea-63118fabef36}, logIndex: 3, isSendImmediate: true 07:08:23,82 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 07:08:23,82 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStates 07:08:23,82 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStates: AppendEntries{term=3, leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:23,82 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=false, member-1-shard-people-oper_testFlipMemberVotingStates=true, member-2-shard-people-oper_testFlipMemberVotingStates=true}, serversVisited=[]], state: Idle 07:08:23,82 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 07:08:23,82 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 07:08:23,82 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-oper_testFlipMemberVotingStates 07:08:23,82 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates, votingState=VOTING]] 07:08:23,82 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-1-shard-cars-oper_testFlipMemberVotingStates: AppendEntries{term=3, leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:23,82 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]] 07:08:23,82 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 07:08:23,82 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=true, member-1-shard-cars-oper_testFlipMemberVotingStates=true, member-3-shard-cars-oper_testFlipMemberVotingStates=false}, serversVisited=[]] 07:08:23,82 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 07:08:23,83 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=50c7582a-2015-4e10-a2ea-b71e6fcdbaab}, logIndex: 3, isSendImmediate: true 07:08:23,83 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 07:08:23,83 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 07:08:23,83 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-2-shard-people-oper_testFlipMemberVotingStates 07:08:23,83 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-cars-oper_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 07:08:23,83 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard cars 07:08:23,83 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 07:08:23,83 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStates: AppendEntries{term=3, leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:23,83 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 07:08:23,83 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 07:08:23,83 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 07:08:23,83 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-people-oper_testFlipMemberVotingStates 07:08:23,83 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-1-shard-people-oper_testFlipMemberVotingStates: AppendEntries{term=3, leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:23,83 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=false, member-1-shard-people-oper_testFlipMemberVotingStates=true, member-2-shard-people-oper_testFlipMemberVotingStates=true}, serversVisited=[]] 07:08:23,83 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-people-oper_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 07:08:23,83 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard people 07:08:23,83 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 07:08:23,83 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 07:08:23,83 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-440670590] 07:08:23,84 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:08:23,84 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-251673576] 07:08:23,84 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-251673576] 07:08:23,84 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 07:08:23,84 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 07:08:23,86 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:23,86 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 07:08:23,86 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 07:08:23,86 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 07:08:23,86 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 07:08:23,86 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates, votingState=VOTING]] 07:08:23,86 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 2 07:08:23,86 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:23,86 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:23,86 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 07:08:23,86 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 07:08:23,86 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 07:08:23,86 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 07:08:23,86 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:23,86 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 07:08:23,86 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 07:08:23,86 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 07:08:23,86 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 07:08:23,86 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates, votingState=NON_VOTING]] 07:08:23,87 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 2 07:08:23,87 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:23,87 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates, votingState=NON_VOTING]] 07:08:23,87 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 2 07:08:23,87 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:23,87 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:23,87 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 07:08:23,87 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 07:08:23,87 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 07:08:23,87 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 07:08:23,87 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates, votingState=VOTING]] 07:08:23,87 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 07:08:23,87 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 2 07:08:23,87 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:23,87 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 07:08:23,88 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 07:08:23,88 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 07:08:23,90 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 07:08:23,90 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 07:08:23,90 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Setting last applied to 3 07:08:23,90 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 07:08:23,90 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 2 to 3 07:08:23,90 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 07:08:23,90 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Setting last applied to 3 07:08:23,90 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 07:08:23,90 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 2 to 3 07:08:23,90 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 07:08:23,90 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 07:08:23,90 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 07:08:23,90 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 07:08:23,90 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 07:08:23,90 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Setting last applied to 3 07:08:23,90 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 2 to 3 07:08:23,90 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 07:08:23,90 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,90 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,90 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 07:08:23,90 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,90 AM [cluster-test-shard-dispatcher-33] [INFO] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 07:08:23,90 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 07:08:23,90 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Initiating leader transfer 07:08:23,90 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 07:08:23,91 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 07:08:23,91 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Initiating leader transfer 07:08:23,91 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2552695c 07:08:23,91 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2552695c 07:08:23,91 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@ff1dee5 07:08:23,91 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 07:08:23,91 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-3-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@ff1dee5 07:08:23,91 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 07:08:23,91 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 07:08:23,91 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 07:08:23,91 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,91 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,91 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,91 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,91 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-oper_testFlipMemberVotingStates 07:08:23,91 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-people-oper_testFlipMemberVotingStates 07:08:23,90 AM [cluster-test-shard-dispatcher-35] [INFO] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 07:08:23,91 AM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 07:08:23,91 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Initiating leader transfer 07:08:23,91 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-1-shard-people-oper_testFlipMemberVotingStates: AppendEntries{term=3, leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:23,91 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-1-shard-cars-oper_testFlipMemberVotingStates: AppendEntries{term=3, leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:23,91 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 07:08:23,92 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 07:08:23,92 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@73059485 07:08:23,92 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 07:08:23,92 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardDataTree - member-3-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@73059485 07:08:23,92 AM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 07:08:23,92 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 07:08:23,92 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 07:08:23,92 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Setting last applied to 3 07:08:23,92 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 07:08:23,92 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 2 to 3 07:08:23,92 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,92 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 07:08:23,92 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 07:08:23,92 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 07:08:23,92 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,92 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,92 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-config_testFlipMemberVotingStates 07:08:23,92 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:23,92 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-1-shard-cars-config_testFlipMemberVotingStates: AppendEntries{term=3, leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:23,92 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 07:08:23,92 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorServerConfigurationSupport - member-3-shard-people-config_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 07:08:23,92 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 07:08:23,92 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 07:08:23,92 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Initiating leader transfer 07:08:23,92 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 07:08:23,92 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 07:08:23,92 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 07:08:23,92 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates, votingState=NON_VOTING]] 07:08:23,92 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 2 07:08:23,92 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@bce5bc3 07:08:23,92 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 07:08:23,92 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-3-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@bce5bc3 07:08:23,92 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=member-1-shard-people-config_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:23,92 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 07:08:23,93 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 07:08:23,93 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 07:08:23,93 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,93 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-people-config_testFlipMemberVotingStates 07:08:23,93 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-1-shard-people-config_testFlipMemberVotingStates: AppendEntries{term=3, leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:23,93 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,93 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 07:08:23,93 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 07:08:23,94 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:23,94 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 07:08:23,94 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 07:08:23,94 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 07:08:23,94 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 07:08:23,94 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates, votingState=VOTING]] 07:08:23,94 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 2 07:08:23,94 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=member-1-shard-cars-config_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:23,94 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:23,94 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 07:08:23,94 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 07:08:23,94 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 07:08:23,94 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 07:08:23,94 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:23,95 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 07:08:23,95 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates, votingState=NON_VOTING]] 07:08:23,95 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 07:08:23,95 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 2 07:08:23,95 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=member-1-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:23,95 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 07:08:23,95 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 07:08:23,95 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates, votingState=NON_VOTING]] 07:08:23,95 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 2 07:08:23,95 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=member-1-shard-people-oper_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:23,95 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 07:08:23,95 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 07:08:23,95 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 07:08:23,96 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 07:08:23,96 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,96 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-1-shard-people-config_testFlipMemberVotingStates, matchIndex: 3, lastIndex: 3, isVoting: true 07:08:23,96 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 07:08:23,96 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,96 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,96 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Leader transfer complete 07:08:23,96 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 07:08:23,99 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Commit index set to 3 07:08:23,99 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 07:08:23,99 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 3 07:08:23,99 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 2 to 3 07:08:23,99 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 07:08:23,99 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 07:08:23,99 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 07:08:23,99 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Commit index set to 3 07:08:23,100 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 07:08:23,100 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 3 07:08:23,100 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 2 to 3 07:08:23,100 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 07:08:23,100 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 07:08:23,100 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 07:08:23,100 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Commit index set to 3 07:08:23,100 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 07:08:23,100 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 3 07:08:23,100 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 2 to 3 07:08:23,100 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 07:08:23,100 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 07:08:23,100 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 07:08:23,100 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates] 07:08:23,100 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 07:08:23,100 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates] 07:08:23,100 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 07:08:23,101 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates] 07:08:23,101 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 07:08:23,102 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Commit index set to 3 07:08:23,102 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 07:08:23,102 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 3 07:08:23,102 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 2 to 3 07:08:23,102 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 07:08:23,102 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 07:08:23,102 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 07:08:23,102 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-people-config_testFlipMemberVotingStates] 07:08:23,102 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 07:08:23,103 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 07:08:23,103 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,103 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-1-shard-cars-config_testFlipMemberVotingStates, matchIndex: 3, lastIndex: 3, isVoting: true 07:08:23,103 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 07:08:23,103 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,103 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,103 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Leader transfer complete 07:08:23,103 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 07:08:23,103 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 07:08:23,103 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,103 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-1-shard-cars-oper_testFlipMemberVotingStates, matchIndex: 3, lastIndex: 3, isVoting: true 07:08:23,103 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 07:08:23,104 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,104 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,104 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Leader transfer complete 07:08:23,104 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 07:08:23,104 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 07:08:23,104 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,104 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-1-shard-people-oper_testFlipMemberVotingStates, matchIndex: 3, lastIndex: 3, isVoting: true 07:08:23,104 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 07:08:23,104 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,104 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,105 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Leader transfer complete 07:08:23,105 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 07:08:23,107 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates] 07:08:23,108 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 07:08:23,108 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates] 07:08:23,108 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 07:08:23,108 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:23,108 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 07:08:23,108 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): matchEntry SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] is present: existingEntryTerm: 3 07:08:23,108 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 3, entries to be added from: 1 07:08:23,108 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 2 07:08:23,108 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Commit index set to 3 07:08:23,108 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=member-1-shard-people-oper_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:23,108 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 07:08:23,108 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 3 07:08:23,108 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 2 to 3 07:08:23,108 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 07:08:23,108 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:23,108 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 07:08:23,108 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 07:08:23,109 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 07:08:23,109 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): matchEntry SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] is present: existingEntryTerm: 3 07:08:23,109 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 3, entries to be added from: 1 07:08:23,109 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 2 07:08:23,109 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Commit index set to 3 07:08:23,109 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=member-1-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:23,109 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates] 07:08:23,109 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 07:08:23,109 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 3 07:08:23,109 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 2 to 3 07:08:23,109 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 07:08:23,109 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 07:08:23,109 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 07:08:23,109 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:23,109 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 07:08:23,109 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): matchEntry SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] is present: existingEntryTerm: 3 07:08:23,109 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 3, entries to be added from: 1 07:08:23,109 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 2 07:08:23,109 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Commit index set to 3 07:08:23,109 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=member-1-shard-cars-config_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:23,109 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 07:08:23,109 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 3 07:08:23,109 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 2 to 3 07:08:23,109 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 07:08:23,109 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 07:08:23,109 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 07:08:23,109 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]} 07:08:23,109 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 07:08:23,109 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): matchEntry SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] is present: existingEntryTerm: 3 07:08:23,110 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 3, entries to be added from: 1 07:08:23,110 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 2 07:08:23,110 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Commit index set to 3 07:08:23,110 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=member-1-shard-people-config_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:23,110 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 07:08:23,110 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 3 07:08:23,110 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 2 to 3 07:08:23,110 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 07:08:23,110 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 07:08:23,110 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 07:08:23,110 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-config_testFlipMemberVotingStates] 07:08:23,110 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 4 07:08:23,110 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Sending RequestVote{term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3} to peer member-2-shard-people-config_testFlipMemberVotingStates 07:08:23,110 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 07:08:23,110 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 07:08:23,110 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 07:08:23,110 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:23,110 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testFlipMemberVotingStates] 07:08:23,110 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 4 07:08:23,110 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 07:08:23,110 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Sending RequestVote{term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3} to peer member-2-shard-cars-config_testFlipMemberVotingStates 07:08:23,110 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 07:08:23,110 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 07:08:23,110 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 07:08:23,110 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 07:08:23,111 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 07:08:23,111 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:23,111 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 07:08:23,111 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 07:08:23,111 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 07:08:23,111 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 07:08:23,111 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-oper_testFlipMemberVotingStates] 07:08:23,111 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 4 07:08:23,111 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Sending RequestVote{term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3} to peer member-2-shard-cars-oper_testFlipMemberVotingStates 07:08:23,111 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 07:08:23,111 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 07:08:23,111 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:23,111 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 07:08:23,111 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 07:08:23,111 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 07:08:23,111 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-people-config_testFlipMemberVotingStates] 07:08:23,111 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 07:08:23,111 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 07:08:23,111 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-oper_testFlipMemberVotingStates] 07:08:23,111 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 4 07:08:23,112 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Sending RequestVote{term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3} to peer member-2-shard-people-oper_testFlipMemberVotingStates 07:08:23,112 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 07:08:23,112 AM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 07:08:23,112 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:23,112 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 07:08:23,112 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 07:08:23,112 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 07:08:23,114 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:08:23,114 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Cluster state: [] 07:08:23,114 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote{term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3} with higher term appears reachable, updating term. 07:08:23,114 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than follower's term 3 - updating term 07:08:23,114 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): In requestVote: RequestVote{term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3} - currentTerm: 4, votedFor: null, lastIndex: 3, lastTerm: 3 07:08:23,114 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply{term=4, voteGranted=true} 07:08:23,114 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:08:23,114 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Cluster state: [] 07:08:23,114 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote{term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3} with higher term appears reachable, updating term. 07:08:23,114 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than follower's term 3 - updating term 07:08:23,114 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): In requestVote: RequestVote{term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3} - currentTerm: 4, votedFor: null, lastIndex: 3, lastTerm: 3 07:08:23,114 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:08:23,114 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply{term=4, voteGranted=true} 07:08:23,114 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Cluster state: [] 07:08:23,114 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote{term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3} with higher term appears reachable, updating term. 07:08:23,114 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than follower's term 3 - updating term 07:08:23,114 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): In requestVote: RequestVote{term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3} - currentTerm: 4, votedFor: null, lastIndex: 3, lastTerm: 3 07:08:23,114 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:08:23,114 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply{term=4, voteGranted=true} 07:08:23,114 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Cluster state: [] 07:08:23,114 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote{term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3} with higher term appears reachable, updating term. 07:08:23,114 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than follower's term 3 - updating term 07:08:23,114 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): In requestVote: RequestVote{term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3} - currentTerm: 4, votedFor: null, lastIndex: 3, lastTerm: 3 07:08:23,114 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply{term=4, voteGranted=true} 07:08:23,116 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Cannot append entries because sender's term 3 is less than 4 07:08:23,116 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Cannot append entries because sender's term 3 is less than 4 07:08:23,116 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Cannot append entries because sender's term 3 is less than 4 07:08:23,116 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply{term=4, voteGranted=true}, my term is 4 07:08:23,117 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply{term=4, voteGranted=true}, current voteCount: 1 07:08:23,117 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-people-config_testFlipMemberVotingStates, member-3-shard-people-config_testFlipMemberVotingStates] 07:08:23,117 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply{term=4, voteGranted=true}, my term is 4 07:08:23,117 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,117 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply{term=4, voteGranted=true}, current voteCount: 1 07:08:23,117 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,117 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-3-shard-cars-config_testFlipMemberVotingStates, member-2-shard-cars-config_testFlipMemberVotingStates] 07:08:23,117 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 07:08:23,117 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 07:08:23,117 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,117 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 07:08:23,117 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:23,122 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to Leader 07:08:23,122 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Candidate to Leader 07:08:23,117 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply{term=4, voteGranted=true}, my term is 4 07:08:23,118 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply{term=4, voteGranted=true}, my term is 4 07:08:23,119 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done false 07:08:23,123 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply{term=4, voteGranted=true}, current voteCount: 1 07:08:23,123 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply{term=4, voteGranted=true}, current voteCount: 1 07:08:23,123 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStates, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:23,123 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-people-oper_testFlipMemberVotingStates, member-3-shard-people-oper_testFlipMemberVotingStates] 07:08:23,123 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-cars-oper_testFlipMemberVotingStates, member-3-shard-cars-oper_testFlipMemberVotingStates] 07:08:23,122 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,123 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 07:08:23,123 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,123 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,123 AM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 07:08:23,123 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 07:08:23,123 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to Leader 07:08:23,123 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:23,123 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Candidate to Leader 07:08:23,123 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 07:08:23,123 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,123 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,123 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 07:08:23,123 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 07:08:23,123 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 07:08:23,123 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 07:08:23,123 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 07:08:23,123 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 07:08:23,123 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:23,123 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 07:08:23,123 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:23,123 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 07:08:23,123 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Candidate to Leader 07:08:23,123 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Candidate to Leader 07:08:23,123 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 07:08:23,124 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 07:08:23,124 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,125 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntriesReply{term=4, success=false, followerId=member-2-shard-cars-config_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}" message is greater than leader's term 3 - switching to Follower 07:08:23,125 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 07:08:23,125 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 07:08:23,125 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:23,125 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from Leader to Follower 07:08:23,125 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntriesReply{term=4, success=false, followerId=member-2-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}" message is greater than leader's term 3 - switching to Follower 07:08:23,125 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 07:08:23,125 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 07:08:23,125 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:23,125 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from Leader to Follower 07:08:23,125 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 07:08:23,125 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 07:08:23,125 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from Leader to Follower 07:08:23,125 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from Leader to Follower 07:08:23,127 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntries{term=4, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 3 - switching to Follower 07:08:23,127 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntriesReply{term=4, success=false, followerId=member-2-shard-people-oper_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}" message is greater than leader's term 3 - switching to Follower 07:08:23,127 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 07:08:23,127 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 07:08:23,127 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:23,127 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 07:08:23,127 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 07:08:23,127 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:23,127 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from Leader to Follower 07:08:23,127 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testFlipMemberVotingStates: leader changed to member-1-shard-cars-config_testFlipMemberVotingStates 07:08:23,127 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 07:08:23,127 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from Leader to Follower 07:08:23,127 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from Leader to Follower 07:08:23,127 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 07:08:23,127 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from Leader to Follower 07:08:23,127 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:23,127 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testFlipMemberVotingStates: Successfully transferred leadership to member-1-shard-cars-config_testFlipMemberVotingStates in 36.00 ms 07:08:23,128 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: leader transfer succeeded after change to non-voting 07:08:23,128 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStates status sync done false 07:08:23,128 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-oper_testFlipMemberVotingStates: leader changed to member-1-shard-people-oper_testFlipMemberVotingStates 07:08:23,128 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorLeadershipTransferCohort - member-3-shard-people-oper_testFlipMemberVotingStates: Successfully transferred leadership to member-1-shard-people-oper_testFlipMemberVotingStates in 37.27 ms 07:08:23,128 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: leader transfer succeeded after change to non-voting 07:08:23,128 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStates status sync done false 07:08:23,128 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-oper_testFlipMemberVotingStates: leader changed to member-1-shard-cars-oper_testFlipMemberVotingStates 07:08:23,128 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorLeadershipTransferCohort - member-3-shard-cars-oper_testFlipMemberVotingStates: Successfully transferred leadership to member-1-shard-cars-oper_testFlipMemberVotingStates in 37.67 ms 07:08:23,128 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: leader transfer succeeded after change to non-voting 07:08:23,128 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStates status sync done false 07:08:23,129 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:23,129 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 07:08:23,129 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done false 07:08:23,129 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done false 07:08:23,129 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:23,129 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:23,129 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 07:08:23,129 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done false 07:08:23,129 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:23,129 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 07:08:23,129 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done true 07:08:23,129 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 07:08:23,131 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 07:08:23,131 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 07:08:23,131 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,131 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 07:08:23,131 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,131 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,133 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done true 07:08:23,133 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done true 07:08:23,133 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done true 07:08:23,134 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-251673576] 07:08:23,135 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-440670590] 07:08:23,135 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 07:08:23,135 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#1813337631] 07:08:23,135 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#1813337631] 07:08:23,135 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1680659489] 07:08:23,135 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 07:08:23,135 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#146984980] 07:08:23,135 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#146984980] 07:08:23,136 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1680659489] 07:08:23,136 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 07:08:23,136 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#-8331463] 07:08:23,136 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#-8331463] 07:08:23,136 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#10783124] 07:08:23,136 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 07:08:23,137 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#-571504725] 07:08:23,137 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#-571504725] 07:08:23,137 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 07:08:23,137 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#10783124] 07:08:23,137 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 07:08:23,137 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#-901496600] 07:08:23,137 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,137 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#-901496600] 07:08:23,137 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-870829185] 07:08:23,137 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 07:08:23,137 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#1284105527] 07:08:23,137 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#1284105527] 07:08:23,137 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-870829185] 07:08:23,137 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 07:08:23,138 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#1464419475] 07:08:23,138 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#1464419475] 07:08:23,138 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 07:08:23,138 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,138 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-788522946] 07:08:23,138 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 07:08:23,138 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#-147071551] 07:08:23,138 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 07:08:23,138 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#-147071551] 07:08:23,138 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:23,138 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-788522946] 07:08:23,138 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 07:08:23,139 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#-1574525536] 07:08:23,139 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#-1574525536] 07:08:23,139 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-1192832717] 07:08:23,139 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 07:08:23,139 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#-1586243575] 07:08:23,139 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#-1586243575] 07:08:23,139 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-1192832717] 07:08:23,139 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStates status sync done true 07:08:23,139 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 07:08:23,140 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#-330172143] 07:08:23,140 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#-330172143] 07:08:23,140 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-440670590] 07:08:23,140 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:08:23,140 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-251673576] 07:08:23,140 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-251673576] 07:08:23,141 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 07:08:23,141 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStates status sync done true 07:08:23,141 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStates status sync done true 07:08:23,141 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testFlipMemberVotingStates 07:08:23,141 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testFlipMemberVotingStates 07:08:23,141 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 07:08:23,141 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onShutDown 07:08:23,142 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onShutDown 07:08:23,142 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testFlipMemberVotingStates 07:08:23,143 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - Stopping Shard member-3-shard-people-config_testFlipMemberVotingStates 07:08:23,143 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 07:08:23,143 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 07:08:23,144 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 07:08:23,144 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testFlipMemberVotingStates 07:08:23,144 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testFlipMemberVotingStates 07:08:23,144 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 07:08:23,144 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onShutDown 07:08:23,145 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onShutDown 07:08:23,145 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testFlipMemberVotingStates 07:08:23,145 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testFlipMemberVotingStates 07:08:23,146 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 07:08:23,146 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 07:08:23,151 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:08:23,151 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:08:23,161 AM [cluster-test-pekko.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:08:23,161 AM [cluster-test-pekko.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:08:23,162 AM [cluster-test-pekko.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:08:23,162 AM [cluster-test-pekko.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:08:23,166 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 07:08:23,175 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 07:08:23,175 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testFlipMemberVotingStates 07:08:23,176 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testFlipMemberVotingStates 07:08:23,176 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 07:08:23,176 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onShutDown 07:08:23,176 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onShutDown 07:08:23,176 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testFlipMemberVotingStates 07:08:23,177 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Stopping Shard member-2-shard-people-config_testFlipMemberVotingStates 07:08:23,177 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 07:08:23,177 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 07:08:23,177 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 07:08:23,178 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testFlipMemberVotingStates 07:08:23,178 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testFlipMemberVotingStates 07:08:23,178 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 07:08:23,178 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onShutDown 07:08:23,178 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onShutDown 07:08:23,178 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testFlipMemberVotingStates 07:08:23,182 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testFlipMemberVotingStates 07:08:23,182 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 07:08:23,182 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 07:08:23,186 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:08:23,186 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:08:23,503 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 07:08:24,172 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-2555493675425138088] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 07:08:24,202 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-356332982335145336] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 07:08:24,204 AM [cluster-test-pekko.actor.default-dispatcher-33] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:08:24,205 AM [cluster-test-pekko.actor.default-dispatcher-33] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:08:24,213 AM [cluster-test-pekko.actor.default-dispatcher-33] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [01/31/2025 07:08:24.216] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:08:24,229 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 07:08:24,230 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testFlipMemberVotingStates 07:08:24,230 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testFlipMemberVotingStates 07:08:24,230 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 07:08:24,230 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onShutDown 07:08:24,230 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Initiating leader transfer 07:08:24,230 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onShutDown 07:08:24,230 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Initiating leader transfer 07:08:24,231 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@361b6585 07:08:24,231 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@361b6585 07:08:24,231 AM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 07:08:24,231 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 07:08:24,231 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 07:08:24,231 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3b42d01f 07:08:24,231 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 07:08:24,231 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3b42d01f 07:08:24,231 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:24,231 AM [cluster-test-shard-dispatcher-10] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 07:08:24,231 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 07:08:24,231 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:24,231 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:24,231 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:25,272 AM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: leader transfer aborted 07:08:25,272 AM [cluster-test-shard-dispatcher-32] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: Failed to transfer leadership in 1.042 s 07:08:25,272 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: In unpauseLeader 07:08:25,272 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: leader transfer failed - sending PoisonPill 07:08:25,272 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Leadership transfer expired 07:08:25,273 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-1-shard-people-config_testFlipMemberVotingStates 07:08:25,283 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: leader transfer aborted 07:08:25,283 AM [cluster-test-shard-dispatcher-34] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: Failed to transfer leadership in 1.053 s 07:08:25,283 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: In unpauseLeader 07:08:25,283 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: leader transfer failed - sending PoisonPill 07:08:25,283 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Leadership transfer expired 07:08:25,284 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testFlipMemberVotingStates 07:08:25,284 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 07:08:25,284 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 07:08:25,285 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 07:08:25,285 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testFlipMemberVotingStates 07:08:25,286 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testFlipMemberVotingStates 07:08:25,286 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 07:08:25,286 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onShutDown 07:08:25,286 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Initiating leader transfer 07:08:25,286 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onShutDown 07:08:25,286 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Initiating leader transfer 07:08:25,286 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@38dc8d36 07:08:25,286 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@38dc8d36 07:08:25,286 AM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 07:08:25,286 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 07:08:25,287 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:25,287 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:25,287 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 07:08:25,287 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 07:08:25,288 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@39ef4d27 07:08:25,288 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@39ef4d27 07:08:25,288 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 07:08:25,288 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 07:08:25,288 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:25,288 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:08:26,362 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: leader transfer aborted 07:08:26,362 AM [cluster-test-shard-dispatcher-10] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: leader transfer aborted 07:08:26,362 AM [cluster-test-shard-dispatcher-10] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: Failed to transfer leadership in 1.076 s 07:08:26,362 AM [cluster-test-shard-dispatcher-34] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: Failed to transfer leadership in 1.076 s 07:08:26,362 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: In unpauseLeader 07:08:26,362 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: In unpauseLeader 07:08:26,362 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: leader transfer failed - sending PoisonPill 07:08:26,362 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: leader transfer failed - sending PoisonPill 07:08:26,362 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Leadership transfer expired 07:08:26,362 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Leadership transfer expired 07:08:26,364 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testFlipMemberVotingStates 07:08:26,364 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testFlipMemberVotingStates 07:08:26,364 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 07:08:26,365 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 07:08:26,367 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:08:26,367 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:08:26,375 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [01/31/2025 07:08:26.377] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:08:26,417 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:08:26,431 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [7348583423927287037] 07:08:26,431 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 07:08:26,434 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 07:08:26,434 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 07:08:26,435 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:08:26,436 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:08:26,449 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:26,451 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 07:08:26,451 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 07:08:26,451 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:26,451 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 07:08:26,454 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testModuleShardLeaderMovement 07:08:26,466 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 07:08:26,468 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1879340533] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:26,471 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 07:08:26,471 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testModuleShardLeaderMovement 07:08:26,472 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testModuleShardLeaderMovement 07:08:26,472 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:26,472 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 07:08:26,472 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 07:08:26,474 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 07:08:26,475 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@6308bb0a} 07:08:26,476 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} started 07:08:26,476 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:26,476 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testModuleShardLeaderMovement 07:08:26,479 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 07:08:26,479 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 07:08:26,479 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testModuleShardLeaderMovement on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement on shard member-1-shard-cars-config_testModuleShardLeaderMovement whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1425136787] 07:08:26,479 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testModuleShardLeaderMovement 07:08:26,480 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-cars-config_testModuleShardLeaderMovement, persistent : true 07:08:26,480 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1425136787] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:26,481 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement/shard-member-1:snapshot-read#1565934130] 07:08:26,482 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement/shard-member-1:snapshot-read#1565934130] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:26,482 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testModuleShardLeaderMovement with journal batch size 1 07:08:26,482 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement-notifier#-323926520 created and ready for shard:member-1-shard-cars-config_testModuleShardLeaderMovement 07:08:26,482 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-people-config_testModuleShardLeaderMovement, persistent : true 07:08:26,482 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement-notifier#-323926520] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:26,482 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement#-1708787650] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:26,483 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-people-config_testModuleShardLeaderMovement with journal batch size 1 07:08:26,483 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement/shard-member-1:snapshot-read#-713875159] 07:08:26,484 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement/shard-member-1:snapshot-read#-713875159] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:26,484 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement-notifier#-918890647] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:26,484 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement-notifier#-918890647 created and ready for shard:member-1-shard-people-config_testModuleShardLeaderMovement 07:08:26,484 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 07:08:26,484 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 07:08:26,484 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testModuleShardLeaderMovement on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement on shard member-1-shard-people-config_testModuleShardLeaderMovement whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement#-1708787650] 07:08:26,486 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:26,486 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:26,486 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 07:08:26,486 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testModuleShardLeaderMovement is initialized 07:08:26,486 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:26,486 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:26,486 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 07:08:26,486 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testModuleShardLeaderMovement is initialized 07:08:26,486 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 07:08:26,486 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:26,486 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 07:08:26,487 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [] 07:08:26,487 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 1 07:08:26,487 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:08:26,487 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 07:08:26,487 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:26,487 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from null to Follower 07:08:26,487 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 07:08:26,488 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Received ElectionTimeout 07:08:26,488 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [] 07:08:26,488 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 07:08:26,488 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 07:08:26,488 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 07:08:26,488 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 07:08:26,488 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 07:08:26,488 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 07:08:26,488 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 07:08:26,488 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 07:08:26,488 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:26,488 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 07:08:26,488 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [] 07:08:26,489 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Candidate): Starting new election term 1 07:08:26,489 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:08:26,489 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 07:08:26,489 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:26,489 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , received role change from null to Follower 07:08:26,489 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 07:08:26,489 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Candidate): Received ElectionTimeout 07:08:26,489 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [] 07:08:26,489 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 07:08:26,489 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 07:08:26,489 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: became leader with frontend state for [] 07:08:26,489 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 07:08:26,489 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 07:08:26,490 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testModuleShardLeaderMovement, leaderId=member-1-shard-people-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 07:08:26,490 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testModuleShardLeaderMovement from Candidate to Leader 07:08:26,490 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 07:08:26,526 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1879340533] 07:08:26,527 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:08:26,527 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1425136787] 07:08:26,527 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:26,529 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:26,532 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testModuleShardLeaderMovement 07:08:26,534 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#1012470576] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:26,537 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 07:08:26,537 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 07:08:26,538 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testModuleShardLeaderMovement 07:08:26,538 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testModuleShardLeaderMovement 07:08:26,538 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:26,538 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 07:08:26,538 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 07:08:26,539 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 07:08:26,540 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@15dbc9ab} 07:08:26,540 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} started 07:08:26,540 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:26,540 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testModuleShardLeaderMovement 07:08:26,541 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 07:08:26,541 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 07:08:26,541 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testModuleShardLeaderMovement on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement on shard member-1-shard-cars-oper_testModuleShardLeaderMovement whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#188337992] 07:08:26,541 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testModuleShardLeaderMovement 07:08:26,541 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-oper_testModuleShardLeaderMovement, persistent : true 07:08:26,542 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 07:08:26,542 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 07:08:26,542 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testModuleShardLeaderMovement on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement on shard member-1-shard-people-oper_testModuleShardLeaderMovement whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement#-1528711799] 07:08:26,542 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Shard created : member-1-shard-people-oper_testModuleShardLeaderMovement, persistent : true 07:08:26,542 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testModuleShardLeaderMovement with journal batch size 1 07:08:26,543 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement-notifier#-2136411323 created and ready for shard:member-1-shard-people-oper_testModuleShardLeaderMovement 07:08:26,543 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement/shard-member-1:snapshot-read#1617551141] 07:08:26,543 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement-notifier#-484753357 created and ready for shard:member-1-shard-cars-oper_testModuleShardLeaderMovement 07:08:26,543 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#188337992] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:26,543 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement#-1528711799] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:26,543 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement/shard-member-1:snapshot-read#488505276] 07:08:26,543 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:26,544 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:26,544 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testModuleShardLeaderMovement with journal batch size 1 07:08:26,544 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 07:08:26,544 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:26,544 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 07:08:26,544 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from null to Follower 07:08:26,544 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 07:08:26,544 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [] 07:08:26,544 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testModuleShardLeaderMovement is initialized 07:08:26,544 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Candidate): Starting new election term 1 07:08:26,544 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:08:26,544 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 07:08:26,544 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:26,544 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from Follower to Candidate 07:08:26,544 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:26,545 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:26,545 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 07:08:26,545 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 07:08:26,545 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testModuleShardLeaderMovement is initialized 07:08:26,545 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:26,545 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , received role change from null to Follower 07:08:26,545 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 07:08:26,545 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [] 07:08:26,545 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Candidate): Received ElectionTimeout 07:08:26,545 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Candidate): Starting new election term 1 07:08:26,545 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:08:26,545 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [] 07:08:26,545 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 07:08:26,545 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:26,545 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 07:08:26,545 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , received role change from Follower to Candidate 07:08:26,545 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 07:08:26,545 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Candidate): Received ElectionTimeout 07:08:26,545 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: became leader with frontend state for [] 07:08:26,545 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from Candidate to Leader 07:08:26,545 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [] 07:08:26,545 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 07:08:26,545 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 07:08:26,545 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: became leader with frontend state for [] 07:08:26,545 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , received role change from Candidate to Leader 07:08:26,546 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 07:08:26,546 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 07:08:26,546 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testModuleShardLeaderMovement, leaderId=member-1-shard-people-oper_testModuleShardLeaderMovement, leaderPayloadVersion=13] 07:08:26,546 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testModuleShardLeaderMovement from Candidate to Leader 07:08:26,546 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderPayloadVersion=13] 07:08:26,546 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testModuleShardLeaderMovement from Candidate to Leader 07:08:26,546 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 07:08:26,590 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#1012470576] 07:08:26,591 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 07:08:26,591 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#188337992] 07:08:26,600 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:08:26,633 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [2501632386193567188] 07:08:26,634 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 07:08:26,637 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:08:26,637 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 07:08:26,638 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:08:26,644 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:08:26,660 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:26,663 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:26,666 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testModuleShardLeaderMovement 07:08:26,677 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-1417501128] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:26,678 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 07:08:26,680 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 07:08:26,683 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 07:08:26,684 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@5b8d9e3e} 07:08:26,684 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} started 07:08:26,686 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:26,689 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:26,691 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:26,694 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testModuleShardLeaderMovement 07:08:26,695 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-814668875] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:26,696 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 07:08:26,698 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 07:08:26,704 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 07:08:26,706 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@554a6b3f} 07:08:26,707 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} started 07:08:26,707 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:26,730 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 07:08:26,741 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 07:08:26,760 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 07:08:26,760 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:26,760 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 07:08:26,760 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:26,760 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 07:08:26,766 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [8335827232589227402] 07:08:26,767 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 07:08:26,770 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:08:26,770 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 07:08:26,771 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:08:26,771 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:08:26,787 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:26,792 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:26,795 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testModuleShardLeaderMovement 07:08:26,804 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-2119587304] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:26,811 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 07:08:26,813 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 07:08:26,816 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 07:08:26,817 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@64435cb6} 07:08:26,818 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} started 07:08:26,818 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:26,820 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:26,820 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:26,824 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testModuleShardLeaderMovement 07:08:26,825 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#1404537131] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:26,826 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 07:08:26,831 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 07:08:26,831 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 07:08:26,832 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@6a84ae4e} 07:08:26,832 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} started 07:08:26,833 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:26,847 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 07:08:26,850 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 07:08:26,851 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:26,851 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 07:08:26,851 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:26,851 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 07:08:27,449 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 07:08:27,449 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 07:08:27,450 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:08:27,450 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:08:27,450 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement 07:08:27,450 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 07:08:27,450 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement 07:08:27,450 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-people-oper_testModuleShardLeaderMovement 07:08:27,450 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 07:08:27,450 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-people-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 07:08:27,450 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 07:08:27,450 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-people-config_testModuleShardLeaderMovement 07:08:27,450 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-people-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 07:08:27,450 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 07:08:27,450 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:27,450 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement 07:08:27,450 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:27,450 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 07:08:27,450 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement 07:08:27,450 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-people-oper_testModuleShardLeaderMovement 07:08:27,450 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 07:08:27,450 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-people-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 07:08:27,450 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 07:08:27,450 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-people-config_testModuleShardLeaderMovement 07:08:27,450 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-people-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 07:08:27,450 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 07:08:27,461 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:08:27,461 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 07:08:27,462 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:27,462 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 07:08:27,462 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:08:27,462 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 07:08:27,462 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:27,462 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 07:08:27,685 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:08:27,685 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 07:08:27,685 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:08:27,685 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 07:08:27,685 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:27,685 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 07:08:27,685 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:27,685 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 07:08:27,740 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 07:08:27,741 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 07:08:27,742 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 07:08:27,742 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement] 07:08:27,747 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 07:08:27,747 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1425136787, primaryVersion=13] 07:08:27,752 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-2-shard-cars-config_testModuleShardLeaderMovement, persistent : true 07:08:27,752 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-556347661] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:27,753 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testModuleShardLeaderMovement with journal batch size 1 07:08:27,753 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-notifier#119580860 created and ready for shard:member-2-shard-cars-config_testModuleShardLeaderMovement 07:08:27,753 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-notifier#119580860] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:27,753 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/shard-member-2:snapshot-read#-2113885815] 07:08:27,754 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/shard-member-2:snapshot-read#-2113885815] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:27,754 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1425136787 for shard member-2-shard-cars-config_testModuleShardLeaderMovement 07:08:27,755 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:27,755 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:27,755 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 07:08:27,755 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:27,755 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from null to Follower 07:08:27,755 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 07:08:27,755 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testModuleShardLeaderMovement is initialized 07:08:27,755 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 07:08:27,755 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from null to Follower 07:08:27,756 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onAddServer: AddServer [newServerId=member-2-shard-cars-config_testModuleShardLeaderMovement, newServerAddress=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement, votingMember=true], state: Idle 07:08:27,757 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating AddServer [newServerId=member-2-shard-cars-config_testModuleShardLeaderMovement, newServerAddress=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement, votingMember=true] 07:08:27,757 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 07:08:27,757 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-config_testModuleShardLeaderMovement 07:08:27,757 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: lastSequenceNumber prior to capture: 1 07:08:27,757 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 07:08:27,757 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating capture snapshot for new server member-2-shard-cars-config_testModuleShardLeaderMovement 07:08:27,757 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: CaptureSnapshotReply received by actor 07:08:27,757 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@15785fbe, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testModuleShardLeaderMovement, ServerConfigPayload=null] 07:08:27,757 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 07:08:27,757 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): sendInstallSnapshot 07:08:27,758 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): next snapshot chunk size for follower member-2-shard-cars-config_testModuleShardLeaderMovement: 229 07:08:27,758 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testModuleShardLeaderMovement, member-2-shard-cars-config_testModuleShardLeaderMovement), Chunk: 1/1 07:08:27,758 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 07:08:27,758 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Snapshot success - sequence number: -1 07:08:27,758 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: SnapshotComplete received 07:08:27,758 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: handled a journal response 07:08:27,759 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: handled a snapshot response 07:08:27,759 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 07:08:27,759 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): handleInstallSnapshot: InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]} 07:08:27,760 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 07:08:27,760 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Snapshot received from leader: member-1-shard-cars-config_testModuleShardLeaderMovement 07:08:27,760 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=-1] 07:08:27,760 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 07:08:27,760 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 07:08:27,761 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot on follower: org.opendaylight.controller.cluster.io.FileBackedOutputStream$1@329639b1 07:08:27,761 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Converted InstallSnapshot from leader: member-1-shard-cars-config_testModuleShardLeaderMovement to state 07:08:27,761 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 07:08:27,761 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 07:08:27,761 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Snapshot success - sequence number: 1 07:08:27,761 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from -1 to -1 07:08:27,761 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot 07:08:27,761 AM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 07:08:27,761 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-config_testModuleShardLeaderMovement: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#2036043937] with name member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher 07:08:27,761 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-2-shard-cars-config_testModuleShardLeaderMovement: state snapshot applied in 314.2 μs 07:08:27,762 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Done applying snapshot 07:08:27,762 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): handleInstallSnapshot returning: InstallSnapshotReply{term=1, followerId=member-2-shard-cars-config_testModuleShardLeaderMovement, chunkIndex=1, success=true} 07:08:27,762 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: SnapshotComplete received 07:08:27,762 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: handled a snapshot response 07:08:27,762 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: handled a journal response 07:08:27,763 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#2036043937] 07:08:27,763 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testModuleShardLeaderMovement: publishChanges: rootPath: / 07:08:27,763 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#2036043937] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:27,763 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): handleInstallSnapshotReply: InstallSnapshotReply{term=1, followerId=member-2-shard-cars-config_testModuleShardLeaderMovement, chunkIndex=1, success=true} 07:08:27,764 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-2-shard-cars-config_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to -1, nextIndex set to 0 07:08:27,764 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 07:08:27,764 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-cars-config_testModuleShardLeaderMovement] 07:08:27,764 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]] 07:08:27,764 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 07:08:27,764 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 07:08:27,764 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=a1bc9172-f459-4d5d-92a0-dc1794178033}, logIndex: 0, isSendImmediate: true 07:08:27,764 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:27,765 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testModuleShardLeaderMovement 07:08:27,765 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testModuleShardLeaderMovement: AppendEntries{term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]]]} 07:08:27,765 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: Returning OK for operation AddServer [newServerId=member-2-shard-cars-config_testModuleShardLeaderMovement, newServerAddress=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement, votingMember=true] 07:08:27,763 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher: 175.1 μs 07:08:27,766 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 07:08:27,767 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]]]} 07:08:27,767 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Number of entries to be appended = 1 07:08:27,767 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 07:08:27,767 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 07:08:27,767 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 07:08:27,767 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testModuleShardLeaderMovement} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1425136787 07:08:27,767 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard cars 07:08:27,768 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-config_testModuleShardLeaderMovement 07:08:27,768 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [cars] 07:08:27,768 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 07:08:27,768 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 07:08:27,768 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: Successfully deleted prior snapshots 07:08:27,768 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testModuleShardLeaderMovement, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement, votingState=VOTING]] 07:08:27,769 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Log size is now 1 07:08:27,769 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=member-2-shard-cars-config_testModuleShardLeaderMovement, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:27,769 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 07:08:27,769 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 07:08:27,769 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 07:08:27,770 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 07:08:27,770 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Re-initializing to Follower with previous leaderId member-1-shard-cars-config_testModuleShardLeaderMovement 07:08:27,770 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 07:08:27,770 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 0, nextIndex: 1 07:08:27,770 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:27,770 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 07:08:27,770 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Setting last applied to 0 07:08:27,770 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 07:08:27,770 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from -1 to 0 07:08:27,770 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 07:08:27,770 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:27,770 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] has been successfully replicated to a majority of followers 07:08:27,770 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=0 07:08:27,772 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Commit index set to 0 07:08:27,772 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 07:08:27,772 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Setting last applied to 0 07:08:27,772 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from -1 to 0 07:08:27,772 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 07:08:27,772 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=0 07:08:27,772 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-1417501128] 07:08:27,772 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 07:08:27,772 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-556347661] 07:08:27,772 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-556347661] 07:08:27,773 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 07:08:27,773 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-814668875] 07:08:27,773 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 07:08:27,773 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Local shard cars not found - shards present: [] 07:08:27,774 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 07:08:27,775 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 07:08:27,776 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 07:08:27,777 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement] 07:08:27,780 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 07:08:27,780 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#188337992, primaryVersion=13] 07:08:27,782 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 07:08:27,784 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#188337992 for shard member-2-shard-cars-oper_testModuleShardLeaderMovement 07:08:27,784 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-2-shard-cars-oper_testModuleShardLeaderMovement, persistent : true 07:08:27,785 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement#-1444566586] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:27,786 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testModuleShardLeaderMovement with journal batch size 1 07:08:27,786 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-notifier#655373365 created and ready for shard:member-2-shard-cars-oper_testModuleShardLeaderMovement 07:08:27,786 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/shard-member-2:snapshot-read#1541294592] 07:08:27,786 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-notifier#655373365] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:27,786 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/shard-member-2:snapshot-read#1541294592] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:27,787 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onAddServer: AddServer [newServerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, newServerAddress=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement, votingMember=true], state: Idle 07:08:27,788 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating AddServer [newServerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, newServerAddress=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement, votingMember=true] 07:08:27,788 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 07:08:27,788 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-oper_testModuleShardLeaderMovement 07:08:27,788 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: lastSequenceNumber prior to capture: 1 07:08:27,788 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 07:08:27,788 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating capture snapshot for new server member-2-shard-cars-oper_testModuleShardLeaderMovement 07:08:27,788 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:27,788 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:27,788 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: CaptureSnapshotReply received by actor 07:08:27,789 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 07:08:27,789 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:27,789 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@5a82a95a, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testModuleShardLeaderMovement, ServerConfigPayload=null] 07:08:27,789 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 07:08:27,789 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testModuleShardLeaderMovement , received role change from null to Follower 07:08:27,789 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): sendInstallSnapshot 07:08:27,789 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): next snapshot chunk size for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: 229 07:08:27,789 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-oper_testModuleShardLeaderMovement, member-2-shard-cars-oper_testModuleShardLeaderMovement), Chunk: 1/1 07:08:27,789 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 07:08:27,789 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Snapshot success - sequence number: -1 07:08:27,789 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: SnapshotComplete received 07:08:27,789 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 07:08:27,789 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testModuleShardLeaderMovement is initialized 07:08:27,789 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: handled a snapshot response 07:08:27,789 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: handled a journal response 07:08:27,790 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 07:08:27,790 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testModuleShardLeaderMovement from null to Follower 07:08:27,791 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 07:08:27,791 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleInstallSnapshot: InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]} 07:08:27,792 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 07:08:27,792 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Snapshot received from leader: member-1-shard-cars-oper_testModuleShardLeaderMovement 07:08:27,792 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderPayloadVersion=-1] 07:08:27,792 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 07:08:27,793 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot on follower: org.opendaylight.controller.cluster.io.FileBackedOutputStream$1@5b08ab6f 07:08:27,793 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Converted InstallSnapshot from leader: member-1-shard-cars-oper_testModuleShardLeaderMovement to state 07:08:27,793 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 07:08:27,793 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done false 07:08:27,794 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 07:08:27,794 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Snapshot success - sequence number: 1 07:08:27,794 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from -1 to -1 07:08:27,794 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot 07:08:27,794 AM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 07:08:27,794 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-oper_testModuleShardLeaderMovement: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#243861812] with name member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher 07:08:27,794 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testModuleShardLeaderMovement: state snapshot applied in 488.1 μs 07:08:27,794 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Done applying snapshot 07:08:27,794 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleInstallSnapshot returning: InstallSnapshotReply{term=1, followerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, chunkIndex=1, success=true} 07:08:27,794 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: SnapshotComplete received 07:08:27,795 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: handled a journal response 07:08:27,795 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: handled a snapshot response 07:08:27,795 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#243861812] 07:08:27,795 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#243861812] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:27,795 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testModuleShardLeaderMovement: publishChanges: rootPath: / 07:08:27,795 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher: 76.64 μs 07:08:27,796 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): handleInstallSnapshotReply: InstallSnapshotReply{term=1, followerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, chunkIndex=1, success=true} 07:08:27,796 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-2-shard-cars-oper_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to -1, nextIndex set to 0 07:08:27,796 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 07:08:27,796 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-cars-oper_testModuleShardLeaderMovement] 07:08:27,796 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]] 07:08:27,796 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 07:08:27,796 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 07:08:27,797 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=d859c55a-7c56-4add-a2f4-ccc1a8343bc3}, logIndex: 0, isSendImmediate: true 07:08:27,797 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:27,797 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-oper_testModuleShardLeaderMovement 07:08:27,797 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testModuleShardLeaderMovement: AppendEntries{term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]]]} 07:08:27,797 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: Returning OK for operation AddServer [newServerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, newServerAddress=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement, votingMember=true] 07:08:27,799 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 07:08:27,803 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testModuleShardLeaderMovement} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#188337992 07:08:27,803 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard cars 07:08:27,803 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]]]} 07:08:27,803 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Number of entries to be appended = 1 07:08:27,803 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 07:08:27,803 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 07:08:27,803 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 07:08:27,803 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testModuleShardLeaderMovement, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement, votingState=VOTING]] 07:08:27,803 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Log size is now 1 07:08:27,803 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:27,803 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-oper_testModuleShardLeaderMovement 07:08:27,803 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars] 07:08:27,803 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 07:08:27,803 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 07:08:27,804 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-814668875] 07:08:27,804 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 07:08:27,804 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: Successfully deleted prior snapshots 07:08:27,804 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement#-1444566586] 07:08:27,804 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement#-1444566586] 07:08:27,804 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testModuleShardLeaderMovement updated: matchIndex: 0, nextIndex: 1 07:08:27,804 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 07:08:27,804 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Setting last applied to 0 07:08:27,804 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from -1 to 0 07:08:27,804 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 07:08:27,804 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderPayloadVersion=13] 07:08:27,805 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:27,805 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 07:08:27,805 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] has been successfully replicated to a majority of followers 07:08:27,805 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=0 07:08:27,805 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 07:08:27,808 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 07:08:27,808 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Commit index set to 0 07:08:27,808 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 07:08:27,808 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Setting last applied to 0 07:08:27,808 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from -1 to 0 07:08:27,808 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 07:08:27,809 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 07:08:27,809 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=0 07:08:27,809 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 07:08:27,809 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Re-initializing to Follower with previous leaderId member-1-shard-cars-oper_testModuleShardLeaderMovement 07:08:27,809 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 07:08:27,809 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:27,809 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done true 07:08:27,809 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 07:08:27,810 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement, visitedAddresses: [pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement] 07:08:27,812 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 07:08:27,812 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1425136787, primaryVersion=13] 07:08:27,815 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1425136787 for shard member-3-shard-cars-config_testModuleShardLeaderMovement 07:08:27,815 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-3-shard-cars-config_testModuleShardLeaderMovement, persistent : true 07:08:27,815 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-1448915771] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:27,816 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-notifier#-296456691 created and ready for shard:member-3-shard-cars-config_testModuleShardLeaderMovement 07:08:27,816 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/shard-member-3:snapshot-read#699841829] 07:08:27,816 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-notifier#-296456691] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:27,816 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/shard-member-3:snapshot-read#699841829] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:27,817 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testModuleShardLeaderMovement with journal batch size 1 07:08:27,817 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onAddServer: AddServer [newServerId=member-3-shard-cars-config_testModuleShardLeaderMovement, newServerAddress=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement, votingMember=true], state: Idle 07:08:27,817 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating AddServer [newServerId=member-3-shard-cars-config_testModuleShardLeaderMovement, newServerAddress=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement, votingMember=true] 07:08:27,817 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-config_testModuleShardLeaderMovement 07:08:27,817 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: lastSequenceNumber prior to capture: 3 07:08:27,818 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 07:08:27,818 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating capture snapshot for new server member-3-shard-cars-config_testModuleShardLeaderMovement 07:08:27,818 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:27,818 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:27,818 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: CaptureSnapshotReply received by actor 07:08:27,818 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 07:08:27,818 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testModuleShardLeaderMovement is initialized 07:08:27,818 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@1a870b22, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testModuleShardLeaderMovement, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 07:08:27,818 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 07:08:27,818 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): sendInstallSnapshot 07:08:27,818 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 07:08:27,818 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:27,818 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): next snapshot chunk size for follower member-3-shard-cars-config_testModuleShardLeaderMovement: 229 07:08:27,818 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from null to Follower 07:08:27,818 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testModuleShardLeaderMovement, member-3-shard-cars-config_testModuleShardLeaderMovement), Chunk: 1/1 07:08:27,819 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 07:08:27,819 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Snapshot success - sequence number: -1 07:08:27,819 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: SnapshotComplete received 07:08:27,819 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: handled a snapshot response 07:08:27,819 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: handled a journal response 07:08:27,819 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 07:08:27,819 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from null to Follower 07:08:27,820 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]}" message is greater than follower's term 0 - updating term 07:08:27,820 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): handleInstallSnapshot: InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]} 07:08:27,820 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 07:08:27,820 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Snapshot received from leader: member-1-shard-cars-config_testModuleShardLeaderMovement 07:08:27,820 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=-1] 07:08:27,820 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 07:08:27,821 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 07:08:27,823 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot on follower: org.opendaylight.controller.cluster.io.FileBackedOutputStream$1@6ebb37c6 07:08:27,823 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Converted InstallSnapshot from leader: member-1-shard-cars-config_testModuleShardLeaderMovement to state 07:08:27,823 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 07:08:27,824 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 07:08:27,824 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Snapshot success - sequence number: 1 07:08:27,824 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from -1 to 0 07:08:27,824 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testModuleShardLeaderMovement, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement, votingState=VOTING], PeerInfo [id=member-2-shard-cars-config_testModuleShardLeaderMovement, address=null, votingState=VOTING]] 07:08:27,824 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot 07:08:27,824 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 07:08:27,825 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-config_testModuleShardLeaderMovement: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#10561248] with name member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher 07:08:27,825 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#10561248] 07:08:27,825 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testModuleShardLeaderMovement: publishChanges: rootPath: / 07:08:27,825 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#10561248] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:27,825 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-3-shard-cars-config_testModuleShardLeaderMovement: state snapshot applied in 676.1 μs 07:08:27,825 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Done applying snapshot 07:08:27,825 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): handleInstallSnapshot returning: InstallSnapshotReply{term=1, followerId=member-3-shard-cars-config_testModuleShardLeaderMovement, chunkIndex=1, success=true} 07:08:27,825 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher: 107.0 μs 07:08:27,826 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): handleInstallSnapshotReply: InstallSnapshotReply{term=1, followerId=member-3-shard-cars-config_testModuleShardLeaderMovement, chunkIndex=1, success=true} 07:08:27,827 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: SnapshotComplete received 07:08:27,827 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-3-shard-cars-config_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to 0, nextIndex set to 1 07:08:27,827 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 07:08:27,827 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: handled a snapshot response 07:08:27,827 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: handled a journal response 07:08:27,827 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-3-shard-cars-config_testModuleShardLeaderMovement] 07:08:27,827 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: New server configuration : [ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]] 07:08:27,827 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Persist data SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 07:08:27,827 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 07:08:27,827 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=b57395ae-b2e5-4a06-88dd-04abe21d3897}, logIndex: 1, isSendImmediate: true 07:08:27,827 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:08:27,827 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testModuleShardLeaderMovement 07:08:27,827 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testModuleShardLeaderMovement: AppendEntries{term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]]]} 07:08:27,827 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:08:27,827 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testModuleShardLeaderMovement 07:08:27,827 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testModuleShardLeaderMovement: AppendEntries{term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]]]} 07:08:27,827 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: Returning OK for operation AddServer [newServerId=member-3-shard-cars-config_testModuleShardLeaderMovement, newServerAddress=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement, votingMember=true] 07:08:27,828 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 07:08:27,829 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]]]} 07:08:27,829 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Number of entries to be appended = 1 07:08:27,829 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 07:08:27,829 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 07:08:27,829 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 07:08:27,830 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testModuleShardLeaderMovement, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testModuleShardLeaderMovement, address=null, votingState=VOTING]] 07:08:27,830 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Log size is now 2 07:08:27,830 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=member-2-shard-cars-config_testModuleShardLeaderMovement, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:27,830 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testModuleShardLeaderMovement} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1425136787 07:08:27,830 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully added the replica shard cars 07:08:27,830 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 07:08:27,830 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 07:08:27,831 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-config_testModuleShardLeaderMovement 07:08:27,831 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [cars] 07:08:27,831 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 07:08:27,831 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-2119587304] 07:08:27,832 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 07:08:27,832 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 07:08:27,832 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 07:08:27,832 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-1448915771] 07:08:27,832 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 07:08:27,832 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: Successfully deleted prior snapshots 07:08:27,832 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Setting last applied to 1 07:08:27,832 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from 0 to 1 07:08:27,832 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 07:08:27,832 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-1448915771] 07:08:27,832 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: fakeSnapshot purging log to 0 for term 1 07:08:27,832 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:27,830 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]]]} 07:08:27,834 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Commit index set to 1 07:08:27,834 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Number of entries to be appended = 1 07:08:27,832 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] has been successfully replicated to a majority of followers 07:08:27,834 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 07:08:27,834 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=1 07:08:27,834 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Setting last applied to 1 07:08:27,834 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from 0 to 1 07:08:27,834 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 07:08:27,834 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: fakeSnapshot purging log to 0 for term 1 07:08:27,834 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 07:08:27,834 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=1 07:08:27,834 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 07:08:27,834 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 07:08:27,835 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testModuleShardLeaderMovement, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement, votingState=VOTING], PeerInfo [id=member-2-shard-cars-config_testModuleShardLeaderMovement, address=null, votingState=VOTING]] 07:08:27,835 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Log size is now 1 07:08:27,835 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=member-3-shard-cars-config_testModuleShardLeaderMovement, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:27,835 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 07:08:27,835 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 07:08:27,836 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 07:08:27,836 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 07:08:27,836 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Re-initializing to Follower with previous leaderId member-1-shard-cars-config_testModuleShardLeaderMovement 07:08:27,836 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 07:08:27,836 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:27,836 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done true 07:08:27,836 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#1404537131] 07:08:27,836 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 07:08:27,836 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Local shard cars not found - shards present: [] 07:08:27,836 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 07:08:27,836 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 07:08:27,837 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 07:08:27,837 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement, visitedAddresses: [pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement] 07:08:27,840 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 07:08:27,840 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#188337992, primaryVersion=13] 07:08:27,840 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 07:08:27,840 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:27,842 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Commit index set to 1 07:08:27,842 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 07:08:27,842 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Setting last applied to 1 07:08:27,842 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from 0 to 1 07:08:27,842 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 07:08:27,842 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=1 07:08:27,842 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 07:08:27,844 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#188337992 for shard member-3-shard-cars-oper_testModuleShardLeaderMovement 07:08:27,845 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Shard created : member-3-shard-cars-oper_testModuleShardLeaderMovement, persistent : true 07:08:27,845 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement#1401520734] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:27,846 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/shard-member-3:snapshot-read#2106607330] 07:08:27,846 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-notifier#1246787137 created and ready for shard:member-3-shard-cars-oper_testModuleShardLeaderMovement 07:08:27,846 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-notifier#1246787137] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:27,846 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/shard-member-3:snapshot-read#2106607330] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:27,847 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testModuleShardLeaderMovement with journal batch size 1 07:08:27,847 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onAddServer: AddServer [newServerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, newServerAddress=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement, votingMember=true], state: Idle 07:08:27,847 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating AddServer [newServerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, newServerAddress=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement, votingMember=true] 07:08:27,847 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-oper_testModuleShardLeaderMovement 07:08:27,848 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: lastSequenceNumber prior to capture: 3 07:08:27,848 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:27,848 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 07:08:27,848 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:27,848 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating capture snapshot for new server member-3-shard-cars-oper_testModuleShardLeaderMovement 07:08:27,848 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 07:08:27,848 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 07:08:27,848 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testModuleShardLeaderMovement is initialized 07:08:27,848 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: CaptureSnapshotReply received by actor 07:08:27,848 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:27,848 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testModuleShardLeaderMovement , received role change from null to Follower 07:08:27,848 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@62b4e180, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testModuleShardLeaderMovement, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 07:08:27,848 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 07:08:27,848 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 07:08:27,848 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): sendInstallSnapshot 07:08:27,848 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testModuleShardLeaderMovement from null to Follower 07:08:27,848 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): next snapshot chunk size for follower member-3-shard-cars-oper_testModuleShardLeaderMovement: 229 07:08:27,848 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-oper_testModuleShardLeaderMovement, member-3-shard-cars-oper_testModuleShardLeaderMovement), Chunk: 1/1 07:08:27,848 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 07:08:27,848 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Snapshot success - sequence number: -1 07:08:27,848 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: SnapshotComplete received 07:08:27,848 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: handled a snapshot response 07:08:27,849 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: handled a journal response 07:08:27,850 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]}" message is greater than follower's term 0 - updating term 07:08:27,850 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleInstallSnapshot: InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]} 07:08:27,850 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 07:08:27,850 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Snapshot received from leader: member-1-shard-cars-oper_testModuleShardLeaderMovement 07:08:27,850 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderPayloadVersion=-1] 07:08:27,850 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 07:08:27,851 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot on follower: org.opendaylight.controller.cluster.io.FileBackedOutputStream$1@1a58c964 07:08:27,851 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testModuleShardLeaderMovement status sync done false 07:08:27,851 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Converted InstallSnapshot from leader: member-1-shard-cars-oper_testModuleShardLeaderMovement to state 07:08:27,851 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 07:08:27,851 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 07:08:27,851 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Snapshot success - sequence number: 1 07:08:27,851 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from -1 to 0 07:08:27,851 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-oper_testModuleShardLeaderMovement, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement, votingState=VOTING], PeerInfo [id=member-2-shard-cars-oper_testModuleShardLeaderMovement, address=null, votingState=VOTING]] 07:08:27,851 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot 07:08:27,851 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-3-shard-cars-oper_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 07:08:27,852 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-oper_testModuleShardLeaderMovement: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#801345136] with name member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher 07:08:27,852 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#801345136] 07:08:27,852 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testModuleShardLeaderMovement: state snapshot applied in 443.6 μs 07:08:27,852 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-oper_testModuleShardLeaderMovement: publishChanges: rootPath: / 07:08:27,852 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Done applying snapshot 07:08:27,852 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleInstallSnapshot returning: InstallSnapshotReply{term=1, followerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, chunkIndex=1, success=true} 07:08:27,852 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-oper_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher: 103.5 μs 07:08:27,852 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#801345136] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:27,853 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: SnapshotComplete received 07:08:27,853 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: handled a journal response 07:08:27,853 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: handled a snapshot response 07:08:27,853 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): handleInstallSnapshotReply: InstallSnapshotReply{term=1, followerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, chunkIndex=1, success=true} 07:08:27,853 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-3-shard-cars-oper_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to 0, nextIndex set to 1 07:08:27,853 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 07:08:27,853 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-3-shard-cars-oper_testModuleShardLeaderMovement] 07:08:27,853 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: New server configuration : [ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]] 07:08:27,853 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persist data SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 07:08:27,853 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 07:08:27,853 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=e5114969-b304-4560-a628-081b8c10c433}, logIndex: 1, isSendImmediate: true 07:08:27,854 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testModuleShardLeaderMovement: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:08:27,854 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testModuleShardLeaderMovement 07:08:27,854 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testModuleShardLeaderMovement: AppendEntries{term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]]]} 07:08:27,854 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:08:27,854 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testModuleShardLeaderMovement 07:08:27,854 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testModuleShardLeaderMovement: AppendEntries{term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]]]} 07:08:27,854 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: Returning OK for operation AddServer [newServerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, newServerAddress=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement, votingMember=true] 07:08:27,854 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 07:08:27,855 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]]]} 07:08:27,855 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Number of entries to be appended = 1 07:08:27,855 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 07:08:27,855 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 07:08:27,855 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 07:08:27,855 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testModuleShardLeaderMovement, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testModuleShardLeaderMovement, address=null, votingState=VOTING]] 07:08:27,855 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Log size is now 2 07:08:27,855 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]]]} 07:08:27,855 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:27,855 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Number of entries to be appended = 1 07:08:27,856 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 07:08:27,856 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 07:08:27,856 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 07:08:27,856 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testModuleShardLeaderMovement} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#188337992 07:08:27,856 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully added the replica shard cars 07:08:27,856 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 07:08:27,856 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done false 07:08:27,856 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-oper_testModuleShardLeaderMovement 07:08:27,856 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [cars] 07:08:27,856 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 07:08:27,856 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 07:08:27,856 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Setting last applied to 1 07:08:27,856 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 07:08:27,856 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from 0 to 1 07:08:27,856 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testModuleShardLeaderMovement, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement, votingState=VOTING], PeerInfo [id=member-2-shard-cars-oper_testModuleShardLeaderMovement, address=null, votingState=VOTING]] 07:08:27,857 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Log size is now 1 07:08:27,857 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 07:08:27,857 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#1404537131] 07:08:27,857 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 07:08:27,857 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: fakeSnapshot purging log to 0 for term 1 07:08:27,857 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:27,857 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 07:08:27,857 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement#1401520734] 07:08:27,857 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] has been successfully replicated to a majority of followers 07:08:27,857 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=1 07:08:27,857 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement#1401520734] 07:08:27,857 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: Successfully deleted prior snapshots 07:08:27,858 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:27,858 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Commit index set to 1 07:08:27,858 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 07:08:27,858 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Setting last applied to 1 07:08:27,858 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from 0 to 1 07:08:27,858 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 07:08:27,858 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: fakeSnapshot purging log to 0 for term 1 07:08:27,858 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=1 07:08:27,858 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 07:08:27,858 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done true 07:08:27,858 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:27,859 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderPayloadVersion=13] 07:08:27,859 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 07:08:27,860 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 07:08:27,860 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1879340533] 07:08:27,860 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:08:27,860 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1425136787] 07:08:27,860 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1425136787] 07:08:27,860 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Commit index set to 1 07:08:27,861 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-1417501128] 07:08:27,861 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 07:08:27,861 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-556347661] 07:08:27,862 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-556347661] 07:08:27,862 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-2119587304] 07:08:27,862 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 07:08:27,862 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 07:08:27,862 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-1448915771] 07:08:27,862 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Setting last applied to 1 07:08:27,862 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-1448915771] 07:08:27,863 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from 0 to 1 07:08:27,863 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 07:08:27,863 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=1 07:08:27,863 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testModuleShardLeaderMovement status sync done true 07:08:27,863 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 07:08:27,863 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Re-initializing to Follower with previous leaderId member-1-shard-cars-oper_testModuleShardLeaderMovement 07:08:27,863 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 07:08:27,863 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:27,865 AM [main] [INFO] ClusterAdminRpcService - Moving leader to local node member-1 for shard cars, datastoreType Config 07:08:27,865 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1879340533] 07:08:27,865 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:08:27,866 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1425136787] 07:08:27,867 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.MakeLeaderLocal to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1425136787] 07:08:27,868 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onMakeLeaderLocal received 07:08:27,868 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - Leadership transfer complete 07:08:27,869 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1879340533] 07:08:27,870 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:08:27,870 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1425136787] 07:08:27,870 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1425136787] 07:08:27,873 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-1417501128] 07:08:27,874 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 07:08:27,874 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-556347661] 07:08:27,874 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-556347661] 07:08:27,874 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-2119587304] 07:08:27,874 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 07:08:27,874 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-1448915771] 07:08:27,874 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-1448915771] 07:08:27,875 AM [main] [INFO] ClusterAdminRpcService - Moving leader to local node member-2 for shard cars, datastoreType Config 07:08:27,875 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-1417501128] 07:08:27,875 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 07:08:27,875 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-556347661] 07:08:27,875 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.MakeLeaderLocal to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-556347661] 07:08:27,875 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onMakeLeaderLocal received 07:08:27,875 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-1-shard-cars-config_testModuleShardLeaderMovement peerAddress = pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 07:08:27,882 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onRequestLeadership RequestLeadership [requestedFollowerId=member-2-shard-cars-config_testModuleShardLeaderMovement, replyTo=Actor[pekko://cluster-test@127.0.0.1:2559/temp/member-2-shard-cars-config_testModuleShardLeaderMovement$e]] 07:08:27,882 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating leader transfer 07:08:27,883 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 07:08:27,883 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@9461d91 07:08:27,883 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@9461d91 07:08:27,883 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 07:08:27,883 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Attempting to transfer leadership 07:08:27,883 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:27,883 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:27,885 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement] 07:08:27,885 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 07:08:27,885 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done true 07:08:27,886 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement] 07:08:27,886 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 07:08:27,887 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-config_testModuleShardLeaderMovement, matchIndex: 1, lastIndex: 1, isVoting: true 07:08:27,887 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Follower's log matches - sending ElectionTimeout 07:08:27,887 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:27,887 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:27,888 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Leader transfer complete 07:08:27,888 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: leader transfer complete - waiting for new leader 07:08:27,889 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 07:08:27,889 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-config_testModuleShardLeaderMovement, member-3-shard-cars-config_testModuleShardLeaderMovement] 07:08:27,889 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 2 07:08:27,889 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): Sending RequestVote{term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1} to peer member-1-shard-cars-config_testModuleShardLeaderMovement 07:08:27,889 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): Sending RequestVote{term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1} to peer member-3-shard-cars-config_testModuleShardLeaderMovement 07:08:27,889 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 2 07:08:27,889 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 07:08:27,889 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:27,890 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 07:08:27,890 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=-1] 07:08:27,890 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Follower to Candidate 07:08:27,891 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:08:27,891 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Cluster state: [] 07:08:27,891 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Candidate in requestVote:RequestVote{term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1} with higher term appears reachable, updating term. 07:08:27,891 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Term 2 in "RequestVote{term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than leader's term 1 - switching to Follower 07:08:27,891 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Leadership transfer in progress - processing RequestVote 07:08:27,891 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): In requestVote: RequestVote{term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1} - currentTerm: 2, votedFor: null, lastIndex: 1, lastTerm: 1 07:08:27,891 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 07:08:27,891 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:08:27,891 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Cluster state: [] 07:08:27,891 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1} with higher term appears reachable, updating term. 07:08:27,891 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 2 in "RequestVote{term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than follower's term 1 - updating term 07:08:27,891 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): In requestVote: RequestVote{term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1} - currentTerm: 2, votedFor: null, lastIndex: 1, lastTerm: 1 07:08:27,891 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 07:08:27,891 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to Follower, election term: 2 07:08:27,891 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 07:08:27,891 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:27,892 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to Follower 07:08:27,892 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=-1] 07:08:27,892 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Leader to Follower 07:08:27,893 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): RaftRPC message received RequestVoteReply{term=2, voteGranted=true}, my term is 2 07:08:27,893 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): handleRequestVoteReply: RequestVoteReply{term=2, voteGranted=true}, current voteCount: 1 07:08:27,893 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [member-1-shard-cars-config_testModuleShardLeaderMovement, member-3-shard-cars-config_testModuleShardLeaderMovement] 07:08:27,893 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:27,893 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:27,893 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 2 07:08:27,893 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 07:08:27,893 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 07:08:27,893 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 07:08:27,893 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 07:08:27,893 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 07:08:27,893 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 07:08:27,894 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: leader changed to member-2-shard-cars-config_testModuleShardLeaderMovement 07:08:27,894 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 07:08:27,894 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 07:08:27,894 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: Successfully transferred leadership to member-2-shard-cars-config_testModuleShardLeaderMovement in 11.71 ms 07:08:27,894 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Leadership transferred successfully to member-2-shard-cars-config_testModuleShardLeaderMovement 07:08:27,894 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done false 07:08:27,899 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 07:08:27,899 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 07:08:27,899 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 07:08:27,900 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 07:08:27,900 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:27,901 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 07:08:27,901 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:27,901 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ClusterAdminRpcService - Leadership transfer complete 07:08:27,901 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-1417501128] 07:08:27,901 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 07:08:27,901 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-556347661] 07:08:27,902 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-556347661] 07:08:27,902 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done true 07:08:27,903 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done true 07:08:27,903 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1879340533] 07:08:27,903 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:08:27,903 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1425136787] 07:08:27,903 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1425136787] 07:08:27,904 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-2119587304] 07:08:27,904 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 07:08:27,904 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-1448915771] 07:08:27,904 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-1448915771] 07:08:27,904 AM [main] [INFO] ClusterAdminRpcService - Moving leader to local node member-3 for shard cars, datastoreType Config 07:08:27,904 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-2119587304] 07:08:27,904 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 07:08:27,904 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-1448915771] 07:08:27,904 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.MakeLeaderLocal to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-1448915771] 07:08:27,904 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onMakeLeaderLocal received 07:08:27,904 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-2-shard-cars-config_testModuleShardLeaderMovement peerAddress = pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement 07:08:27,906 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onRequestLeadership RequestLeadership [requestedFollowerId=member-3-shard-cars-config_testModuleShardLeaderMovement, replyTo=Actor[pekko://cluster-test@127.0.0.1:2557/temp/member-3-shard-cars-config_testModuleShardLeaderMovement$e]] 07:08:27,906 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Initiating leader transfer 07:08:27,906 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@74589e4b 07:08:27,906 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-2-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@74589e4b 07:08:27,906 AM [cluster-test-shard-dispatcher-14] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 07:08:27,906 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Attempting to transfer leadership 07:08:27,906 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:27,906 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:27,906 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 07:08:27,907 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeaderTransitioning: LeaderTransitioning [leaderId=member-2-shard-cars-config_testModuleShardLeaderMovement] 07:08:27,908 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 07:08:27,908 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): tryToCompleteLeadershipTransfer: followerId: member-3-shard-cars-config_testModuleShardLeaderMovement, matchIndex: 1, lastIndex: 1, isVoting: true 07:08:27,908 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Follower's log matches - sending ElectionTimeout 07:08:27,908 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:27,908 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:27,908 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Leader transfer complete 07:08:27,908 AM [cluster-test-shard-dispatcher-14] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: leader transfer complete - waiting for new leader 07:08:27,909 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeaderTransitioning: LeaderTransitioning [leaderId=member-2-shard-cars-config_testModuleShardLeaderMovement] 07:08:27,909 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 07:08:27,910 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 07:08:27,910 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-config_testModuleShardLeaderMovement, member-2-shard-cars-config_testModuleShardLeaderMovement] 07:08:27,910 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 3 07:08:27,910 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): Sending RequestVote{term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1} to peer member-1-shard-cars-config_testModuleShardLeaderMovement 07:08:27,910 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): Sending RequestVote{term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1} to peer member-2-shard-cars-config_testModuleShardLeaderMovement 07:08:27,911 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 3 07:08:27,911 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 07:08:27,911 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:27,911 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:08:27,911 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 07:08:27,911 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Cluster state: [] 07:08:27,911 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=-1] 07:08:27,911 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1} with higher term appears reachable, updating term. 07:08:27,911 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 3 in "RequestVote{term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than follower's term 2 - updating term 07:08:27,911 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from Follower to Candidate 07:08:27,911 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): In requestVote: RequestVote{term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1} - currentTerm: 3, votedFor: null, lastIndex: 1, lastTerm: 1 07:08:27,911 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:08:27,912 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:08:27,912 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Cluster state: [] 07:08:27,912 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Candidate in requestVote:RequestVote{term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1} with higher term appears reachable, updating term. 07:08:27,912 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Term 3 in "RequestVote{term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than leader's term 2 - switching to Follower 07:08:27,912 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Leadership transfer in progress - processing RequestVote 07:08:27,912 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): In requestVote: RequestVote{term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1} - currentTerm: 3, votedFor: null, lastIndex: 1, lastTerm: 1 07:08:27,912 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:08:27,912 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to Follower, election term: 3 07:08:27,912 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 07:08:27,912 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:27,913 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to Follower 07:08:27,913 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=-1] 07:08:27,913 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Leader to Follower 07:08:27,913 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): RaftRPC message received RequestVoteReply{term=3, voteGranted=true}, my term is 3 07:08:27,913 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): handleRequestVoteReply: RequestVoteReply{term=3, voteGranted=true}, current voteCount: 1 07:08:27,913 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [member-1-shard-cars-config_testModuleShardLeaderMovement, member-2-shard-cars-config_testModuleShardLeaderMovement] 07:08:27,913 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:27,913 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:27,913 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 07:08:27,913 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 07:08:27,913 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 07:08:27,913 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 07:08:27,914 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 07:08:27,914 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 07:08:27,914 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 07:08:27,914 AM [cluster-test-shard-dispatcher-14] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: leader changed to member-3-shard-cars-config_testModuleShardLeaderMovement 07:08:27,914 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: Successfully transferred leadership to member-3-shard-cars-config_testModuleShardLeaderMovement in 8.425 ms 07:08:27,914 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Leadership transferred successfully to member-3-shard-cars-config_testModuleShardLeaderMovement 07:08:27,915 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 07:08:27,915 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 07:08:27,915 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 07:08:27,915 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 07:08:27,915 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done false 07:08:27,915 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 07:08:27,916 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ClusterAdminRpcService - Leadership transfer complete 07:08:27,916 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 07:08:27,916 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:27,916 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 07:08:27,916 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:27,916 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-2119587304] 07:08:27,916 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 07:08:27,916 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-1448915771] 07:08:27,917 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-1448915771] 07:08:27,917 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 07:08:27,917 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 07:08:27,918 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done true 07:08:27,918 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testModuleShardLeaderMovement 07:08:27,919 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onShutDown 07:08:27,919 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Initiating leader transfer 07:08:27,919 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2ebec85 07:08:27,919 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-3-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2ebec85 07:08:27,919 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 07:08:27,919 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Attempting to transfer leadership 07:08:27,919 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:27,919 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:27,919 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): tryToCompleteLeadershipTransfer: followerId: member-1-shard-cars-config_testModuleShardLeaderMovement, matchIndex: 1, lastIndex: 1, isVoting: true 07:08:27,919 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Follower's log matches - sending ElectionTimeout 07:08:27,919 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:27,919 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:27,919 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Leader transfer complete 07:08:27,919 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: leader transfer complete - waiting for new leader 07:08:27,920 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 07:08:27,920 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 07:08:27,920 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-config_testModuleShardLeaderMovement] 07:08:27,920 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 07:08:27,920 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-config_testModuleShardLeaderMovement] 07:08:27,921 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 07:08:27,921 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 07:08:27,921 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-config_testModuleShardLeaderMovement, member-2-shard-cars-config_testModuleShardLeaderMovement] 07:08:27,921 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 4 07:08:27,921 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Sending RequestVote{term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1} to peer member-3-shard-cars-config_testModuleShardLeaderMovement 07:08:27,921 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Sending RequestVote{term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1} to peer member-2-shard-cars-config_testModuleShardLeaderMovement 07:08:27,921 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 4 07:08:27,921 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 07:08:27,921 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:27,921 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 07:08:27,921 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=-1] 07:08:27,921 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Follower to Candidate 07:08:27,923 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:08:27,923 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:08:27,923 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Cluster state: [] 07:08:27,923 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Candidate in requestVote:RequestVote{term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1} with higher term appears reachable, updating term. 07:08:27,923 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than follower's term 3 - updating term 07:08:27,923 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Cluster state: [] 07:08:27,923 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Candidate in requestVote:RequestVote{term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1} with higher term appears reachable, updating term. 07:08:27,923 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than leader's term 3 - switching to Follower 07:08:27,923 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): In requestVote: RequestVote{term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1} - currentTerm: 4, votedFor: null, lastIndex: 1, lastTerm: 1 07:08:27,923 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Leadership transfer in progress - processing RequestVote 07:08:27,923 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): requestVote returning: RequestVoteReply{term=4, voteGranted=true} 07:08:27,923 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): In requestVote: RequestVote{term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1} - currentTerm: 4, votedFor: null, lastIndex: 1, lastTerm: 1 07:08:27,923 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): requestVote returning: RequestVoteReply{term=4, voteGranted=true} 07:08:27,923 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to Follower, election term: 4 07:08:27,923 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 07:08:27,923 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:27,923 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to Follower 07:08:27,924 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=-1] 07:08:27,924 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from Leader to Follower 07:08:27,924 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): RaftRPC message received RequestVoteReply{term=4, voteGranted=true}, my term is 4 07:08:27,924 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): handleRequestVoteReply: RequestVoteReply{term=4, voteGranted=true}, current voteCount: 1 07:08:27,924 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [member-3-shard-cars-config_testModuleShardLeaderMovement, member-2-shard-cars-config_testModuleShardLeaderMovement] 07:08:27,924 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:27,925 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:27,925 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 07:08:27,925 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 07:08:27,925 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 07:08:27,925 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 07:08:27,925 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 07:08:27,925 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 07:08:27,925 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 07:08:27,926 AM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: leader changed to member-1-shard-cars-config_testModuleShardLeaderMovement 07:08:27,926 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 07:08:27,926 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 07:08:27,926 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 07:08:27,926 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 07:08:27,926 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 07:08:27,926 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: Successfully transferred leadership to member-1-shard-cars-config_testModuleShardLeaderMovement in 7.428 ms 07:08:27,927 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 07:08:27,927 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:27,927 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 07:08:27,927 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:27,927 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: leader transfer succeeded - sending PoisonPill 07:08:27,927 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 07:08:27,928 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 07:08:27,929 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testModuleShardLeaderMovement 07:08:27,929 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 07:08:27,929 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 07:08:27,929 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 07:08:27,930 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testModuleShardLeaderMovement 07:08:27,930 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 1 shards 07:08:27,930 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: onShutDown 07:08:27,930 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testModuleShardLeaderMovement 07:08:27,930 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 07:08:27,930 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 07:08:27,932 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:08:27,932 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:08:27,937 AM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:08:27,938 AM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:08:27,938 AM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:08:27,938 AM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:08:27,963 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 07:08:27,964 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testModuleShardLeaderMovement 07:08:27,964 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 1 shards 07:08:27,965 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onShutDown 07:08:27,965 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testModuleShardLeaderMovement 07:08:27,966 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 07:08:27,966 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 07:08:27,967 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 07:08:27,967 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testModuleShardLeaderMovement 07:08:27,967 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 1 shards 07:08:27,967 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: onShutDown 07:08:27,968 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testModuleShardLeaderMovement 07:08:27,968 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 07:08:27,968 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 07:08:27,970 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:08:27,970 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:08:28,468 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 07:08:28,947 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [8335827232589227402] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 07:08:28,987 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [2501632386193567188] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 07:08:28,991 AM [cluster-test-pekko.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:08:28,991 AM [cluster-test-pekko.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:08:28,993 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [01/31/2025 07:08:28.996] [cluster-test-pekko.actor.internal-dispatcher-9] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:08:29,6 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 07:08:29,7 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testModuleShardLeaderMovement 07:08:29,7 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testModuleShardLeaderMovement 07:08:29,7 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 07:08:29,7 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: onShutDown 07:08:29,7 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onShutDown 07:08:29,7 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@75aaf983 07:08:29,7 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating leader transfer 07:08:29,7 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@75aaf983 07:08:29,8 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 07:08:29,8 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@66e6d159 07:08:29,8 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@66e6d159 07:08:29,8 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 07:08:29,8 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Attempting to transfer leadership 07:08:29,8 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:29,8 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-1-shard-people-config_testModuleShardLeaderMovement 07:08:29,8 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:33,568 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Up)]. 07:08:33,568 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2559, Up)]. 07:08:33,569 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:08:33,569 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:33,569 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:08:33,569 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:34,587 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 07:08:35,285 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 07:08:36,566 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 07:08:38,157 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: leader transfer aborted 07:08:38,157 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: Failed to transfer leadership in 9.150 s 07:08:38,158 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: In unpauseLeader 07:08:38,158 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: leader transfer failed - sending PoisonPill 07:08:38,158 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Leadership transfer expired 07:08:38,158 AM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: At least 1 followers need to be active, Switching member-1-shard-cars-config_testModuleShardLeaderMovement from Leader to IsolatedLeader 07:08:38,158 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (IsolatedLeader): Election: Leader has following peers: [member-2-shard-cars-config_testModuleShardLeaderMovement, member-3-shard-cars-config_testModuleShardLeaderMovement] 07:08:38,158 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:38,158 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:38,158 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 4 07:08:38,159 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 07:08:38,159 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to IsolatedLeader 07:08:38,159 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Leader to IsolatedLeader 07:08:38,159 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testModuleShardLeaderMovement 07:08:38,160 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 07:08:38,160 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 07:08:38,160 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 07:08:38,161 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testModuleShardLeaderMovement 07:08:38,161 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testModuleShardLeaderMovement 07:08:38,161 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 07:08:38,161 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: onShutDown 07:08:38,161 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating leader transfer 07:08:38,161 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 07:08:38,162 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: onShutDown 07:08:38,162 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2cf25320 07:08:38,162 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@1632f85f 07:08:38,162 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2cf25320 07:08:38,162 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-people-oper_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@1632f85f 07:08:38,162 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 07:08:38,162 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Attempting to transfer leadership 07:08:38,162 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testModuleShardLeaderMovement: active: false, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:38,162 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: active: false, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:38,162 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testModuleShardLeaderMovement 07:08:41,867 AM [cluster-test-shard-dispatcher-18] [WARN] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: At least 1 followers need to be active, Switching member-1-shard-cars-oper_testModuleShardLeaderMovement from Leader to IsolatedLeader 07:08:41,867 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (IsolatedLeader): Election: Leader has following peers: [member-3-shard-cars-oper_testModuleShardLeaderMovement, member-2-shard-cars-oper_testModuleShardLeaderMovement] 07:08:41,867 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testModuleShardLeaderMovement: active: false, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:41,867 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: active: false, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:08:41,867 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 07:08:41,867 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testModuleShardLeaderMovement: leader transfer aborted 07:08:41,867 AM [cluster-test-shard-dispatcher-18] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testModuleShardLeaderMovement: Failed to transfer leadership in 3.706 s 07:08:41,868 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: In unpauseLeader 07:08:41,868 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: leader transfer failed - sending PoisonPill 07:08:41,868 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 07:08:41,868 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from Leader to IsolatedLeader 07:08:41,868 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testModuleShardLeaderMovement from Leader to IsolatedLeader 07:08:41,869 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testModuleShardLeaderMovement 07:08:41,869 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 07:08:41,869 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 07:08:41,871 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:08:41,871 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:08:41,875 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [01/31/2025 07:08:41.878] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:08:41,915 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 07:08:41,929 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-8658802096870042244] 07:08:41,929 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 07:08:41,931 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 07:08:41,931 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 07:08:41,932 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:08:41,932 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:08:41,947 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:41,949 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 07:08:41,949 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 07:08:41,949 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 07:08:41,952 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:41,956 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 07:08:41,971 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-978735482] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:41,972 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 07:08:41,978 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 07:08:41,978 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testRemoveShardReplica 07:08:41,979 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testRemoveShardReplica 07:08:41,979 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:41,980 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 07:08:41,980 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 07:08:41,980 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} 07:08:41,982 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@30046c5} 07:08:41,982 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} started 07:08:41,983 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:41,984 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:41,984 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testRemoveShardReplica 07:08:41,986 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:41,989 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 07:08:41,990 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 07:08:41,990 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica on shard member-1-shard-cars-config_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#2020213724] 07:08:41,990 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testRemoveShardReplica 07:08:42,7 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-cars-config_testRemoveShardReplica, persistent : true 07:08:42,17 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#2020213724] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:42,18 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 07:08:42,20 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testRemoveShardReplica with journal batch size 1 07:08:42,20 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica-notifier#653225767 created and ready for shard:member-1-shard-cars-config_testRemoveShardReplica 07:08:42,21 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica-notifier#653225767] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:42,21 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#1765865128] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:42,22 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 07:08:42,22 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica/shard-member-1:snapshot-read#-1944003350] 07:08:42,22 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica/shard-member-1:snapshot-read#-1944003350] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:42,23 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-people-config_testRemoveShardReplica, persistent : true 07:08:42,23 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 07:08:42,23 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 07:08:42,23 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica on shard member-1-shard-people-config_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica#705683510] 07:08:42,24 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica#705683510] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:42,27 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 07:08:42,27 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:42,27 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testRemoveShardReplica 07:08:42,27 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:42,28 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testRemoveShardReplica 07:08:42,28 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:42,28 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 07:08:42,28 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 07:08:42,28 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 07:08:42,28 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testRemoveShardReplica is initialized 07:08:42,28 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:08:42,28 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:42,28 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica-notifier#-1067821697 created and ready for shard:member-1-shard-people-config_testRemoveShardReplica 07:08:42,28 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica-notifier#-1067821697] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:42,28 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 07:08:42,29 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardReplica, generation=0} 07:08:42,29 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica/shard-member-1:snapshot-read#1472158102] 07:08:42,29 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica/shard-member-1:snapshot-read#1472158102] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:42,29 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-people-config_testRemoveShardReplica with journal batch size 1 07:08:42,30 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@1f9f727} 07:08:42,30 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardReplica, generation=0} started 07:08:42,31 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:42,31 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testRemoveShardReplica 07:08:42,31 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 07:08:42,31 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 07:08:42,31 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 07:08:42,31 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica on shard member-1-shard-cars-oper_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica#-203832349] 07:08:42,31 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testRemoveShardReplica 07:08:42,31 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-cars-oper_testRemoveShardReplica, persistent : true 07:08:42,31 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica#-203832349] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:42,32 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testRemoveShardReplica with journal batch size 1 07:08:42,32 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica/shard-member-1:snapshot-read#651493170] 07:08:42,32 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica-notifier#1702798274 created and ready for shard:member-1-shard-cars-oper_testRemoveShardReplica 07:08:42,33 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 07:08:42,33 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 07:08:42,33 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica on shard member-1-shard-people-oper_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica#86622819] 07:08:42,34 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:42,34 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:42,37 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:42,37 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:42,37 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:08:42,37 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:42,37 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica-notifier#1702798274] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:42,37 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 07:08:42,37 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 07:08:42,38 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:08:42,38 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testRemoveShardReplica is initialized 07:08:42,38 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:42,38 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from null to Follower 07:08:42,38 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 07:08:42,38 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from null to Follower 07:08:42,37 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-people-oper_testRemoveShardReplica, persistent : true 07:08:42,39 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testRemoveShardReplica with journal batch size 1 07:08:42,39 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica/shard-member-1:snapshot-read#-1522642116] 07:08:42,40 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica-notifier#-1274512972 created and ready for shard:member-1-shard-people-oper_testRemoveShardReplica 07:08:42,41 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from null to Follower 07:08:42,41 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 07:08:42,41 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testRemoveShardReplica is initialized 07:08:42,41 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 07:08:42,42 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from null to Follower 07:08:42,42 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:42,42 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:42,42 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:08:42,42 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:42,42 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 07:08:42,42 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testRemoveShardReplica is initialized 07:08:42,42 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 07:08:42,42 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 07:08:42,42 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from null to Follower 07:08:42,43 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 07:08:42,58 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [5504065899577362811] 07:08:42,58 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 07:08:42,61 AM [cluster-test-pekko.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:08:42,62 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 07:08:42,63 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:08:42,63 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:08:42,79 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:42,81 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:42,84 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 07:08:42,96 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=0} 07:08:42,99 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#1007391554] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:42,100 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 07:08:42,103 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=0} started 07:08:42,103 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 07:08:42,103 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveShardReplica 07:08:42,103 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:42,103 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveShardReplica 07:08:42,104 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@271f5a91} 07:08:42,104 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:42,104 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveShardReplica 07:08:42,105 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:42,105 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveShardReplica 07:08:42,106 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-2-shard-cars-config_testRemoveShardReplica, persistent : true 07:08:42,107 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testRemoveShardReplica with journal batch size 1 07:08:42,107 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-2-shard-people-config_testRemoveShardReplica, persistent : true 07:08:42,107 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 07:08:42,108 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-2-shard-people-config_testRemoveShardReplica with journal batch size 1 07:08:42,109 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 07:08:42,110 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-920540737] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:42,110 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/shard-member-2:snapshot-read#-1049819058] 07:08:42,110 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#-1180260259 created and ready for shard:member-2-shard-people-config_testRemoveShardReplica 07:08:42,110 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/shard-member-2:snapshot-read#-1042316107] 07:08:42,116 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica#869519729] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:42,116 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#-2078230650] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:42,116 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/shard-member-2:snapshot-read#-1049819058] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:42,116 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#-1180260259] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:42,116 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 07:08:42,116 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/shard-member-2:snapshot-read#-1042316107] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:42,116 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveShardReplica 07:08:42,116 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveShardReplica 07:08:42,119 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#1530529271 created and ready for shard:member-2-shard-cars-config_testRemoveShardReplica 07:08:42,119 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#1530529271] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:42,119 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:42,119 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:42,119 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 07:08:42,119 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveShardReplica is initialized 07:08:42,120 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:08:42,120 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:42,120 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , received role change from null to Follower 07:08:42,120 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 07:08:42,120 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveShardReplica from null to Follower 07:08:42,120 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=0} 07:08:42,122 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@1b71871e} 07:08:42,122 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=0} started 07:08:42,122 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:42,122 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveShardReplica 07:08:42,123 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:42,123 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:42,123 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:08:42,123 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveShardReplica 07:08:42,123 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:42,123 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 07:08:42,123 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveShardReplica is initialized 07:08:42,123 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-2-shard-cars-oper_testRemoveShardReplica, persistent : true 07:08:42,124 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testRemoveShardReplica with journal batch size 1 07:08:42,124 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-2-shard-people-oper_testRemoveShardReplica, persistent : true 07:08:42,125 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 07:08:42,125 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 07:08:42,125 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica#7846392] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:42,125 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica#-77380335] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:42,125 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica-notifier#965710245 created and ready for shard:member-2-shard-cars-oper_testRemoveShardReplica 07:08:42,126 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/shard-member-2:snapshot-read#832902184] 07:08:42,126 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica-notifier#714071087 created and ready for shard:member-2-shard-people-oper_testRemoveShardReplica 07:08:42,126 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/shard-member-2:snapshot-read#-1675899875] 07:08:42,126 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardReplica from null to Follower 07:08:42,127 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:42,127 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:42,127 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:08:42,127 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:42,128 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testRemoveShardReplica with journal batch size 1 07:08:42,128 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 07:08:42,128 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 07:08:42,128 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveShardReplica is initialized 07:08:42,129 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 07:08:42,129 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveShardReplica from null to Follower 07:08:42,131 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:42,131 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:42,131 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 07:08:42,132 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveShardReplica is initialized 07:08:42,132 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:08:42,132 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:42,132 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 07:08:42,132 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 07:08:42,132 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveShardReplica from null to Follower 07:08:42,152 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 07:08:42,153 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:08:42,160 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 07:08:42,161 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:42,161 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:42,161 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 07:08:42,161 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 07:08:42,161 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 07:08:42,161 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 07:08:42,161 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 07:08:42,161 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 07:08:42,161 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 07:08:42,161 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 07:08:42,161 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 07:08:42,161 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 07:08:42,161 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 07:08:42,161 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 07:08:42,175 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-7387743005627041772] 07:08:42,176 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 07:08:42,178 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:08:42,178 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 07:08:42,180 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:08:42,180 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:08:42,194 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:42,197 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:42,200 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 07:08:42,217 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1158076466] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:42,217 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 07:08:42,223 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 07:08:42,223 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testRemoveShardReplica 07:08:42,224 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testRemoveShardReplica 07:08:42,225 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} 07:08:42,227 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@609f9fb3} 07:08:42,227 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} started 07:08:42,228 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:42,228 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testRemoveShardReplica 07:08:42,228 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:42,228 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testRemoveShardReplica 07:08:42,229 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-3-shard-cars-config_testRemoveShardReplica, persistent : true 07:08:42,229 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:42,230 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testRemoveShardReplica with journal batch size 1 07:08:42,230 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-3-shard-people-config_testRemoveShardReplica, persistent : true 07:08:42,230 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-3-shard-people-config_testRemoveShardReplica with journal batch size 1 07:08:42,231 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica-notifier#1496559809 created and ready for shard:member-3-shard-cars-config_testRemoveShardReplica 07:08:42,231 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica/shard-member-3:snapshot-read#-368795450] 07:08:42,231 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica-notifier#1358645234 created and ready for shard:member-3-shard-people-config_testRemoveShardReplica 07:08:42,232 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica/shard-member-3:snapshot-read#1064385217] 07:08:42,232 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#1118202765] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:42,233 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 07:08:42,233 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:42,234 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:42,236 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica#1355024857] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:42,240 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 07:08:42,244 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 07:08:42,247 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:08:42,247 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:42,247 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 07:08:42,247 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testRemoveShardReplica is initialized 07:08:42,248 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 07:08:42,248 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:42,248 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 07:08:42,248 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:42,248 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:08:42,248 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:42,248 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardReplica , received role change from null to Follower 07:08:42,251 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:42,251 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 07:08:42,251 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 07:08:42,251 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 07:08:42,251 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 07:08:42,251 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 07:08:42,251 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 07:08:42,251 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 07:08:42,252 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica-notifier#1496559809] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:42,252 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica/shard-member-3:snapshot-read#-368795450] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:42,252 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica-notifier#1358645234] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:42,252 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica/shard-member-3:snapshot-read#1064385217] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:42,252 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#800257462] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:42,253 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 07:08:42,253 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testRemoveShardReplica 07:08:42,253 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testRemoveShardReplica 07:08:42,254 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:42,254 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 07:08:42,254 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 07:08:42,255 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardReplica from null to Follower 07:08:42,255 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 07:08:42,255 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testRemoveShardReplica is initialized 07:08:42,255 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 07:08:42,255 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testRemoveShardReplica from null to Follower 07:08:42,257 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardReplica, generation=0} 07:08:42,258 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@4dfaa06f} 07:08:42,258 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardReplica, generation=0} started 07:08:42,258 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:42,258 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testRemoveShardReplica 07:08:42,258 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} 07:08:42,259 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Shard created : member-3-shard-cars-oper_testRemoveShardReplica, persistent : true 07:08:42,259 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica#-677133965] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:42,259 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica/shard-member-3:snapshot-read#747214657] 07:08:42,260 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica-notifier#-983665036 created and ready for shard:member-3-shard-cars-oper_testRemoveShardReplica 07:08:42,260 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testRemoveShardReplica with journal batch size 1 07:08:42,260 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica-notifier#-983665036] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:42,260 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 07:08:42,260 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 07:08:42,260 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica on shard member-3-shard-cars-oper_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica#-677133965] 07:08:42,260 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testRemoveShardReplica 07:08:42,260 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:42,260 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:42,261 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:08:42,261 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:42,261 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 07:08:42,261 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 07:08:42,261 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-3-shard-people-oper_testRemoveShardReplica, persistent : true 07:08:42,261 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica-notifier#303901969 created and ready for shard:member-3-shard-people-oper_testRemoveShardReplica 07:08:42,261 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testRemoveShardReplica with journal batch size 1 07:08:42,261 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica/shard-member-3:snapshot-read#-1812775152] 07:08:42,261 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 07:08:42,261 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 07:08:42,261 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica on shard member-3-shard-people-oper_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica#-260173350] 07:08:42,262 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 07:08:42,262 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testRemoveShardReplica is initialized 07:08:42,262 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 07:08:42,262 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testRemoveShardReplica from null to Follower 07:08:42,262 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:42,262 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:42,262 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:08:42,262 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:42,263 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 07:08:42,263 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 07:08:42,263 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testRemoveShardReplica is initialized 07:08:42,263 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 07:08:42,263 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 07:08:42,263 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testRemoveShardReplica from null to Follower 07:08:42,355 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 07:08:42,355 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Follower): Received ElectionTimeout - switching to Candidate 07:08:42,355 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-config_testRemoveShardReplica, member-2-shard-cars-config_testRemoveShardReplica] 07:08:42,355 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Starting new election term 1 07:08:42,356 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:08:42,356 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:08:42,356 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:42,357 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from Follower to Candidate 07:08:42,357 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from Follower to Candidate 07:08:42,395 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 07:08:42,395 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Follower): Received ElectionTimeout - switching to Candidate 07:08:42,395 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-config_testRemoveShardReplica, member-3-shard-people-config_testRemoveShardReplica] 07:08:42,396 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Starting new election term 1 07:08:42,396 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:08:42,396 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:08:42,396 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:42,396 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from Follower to Candidate 07:08:42,397 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from Follower to Candidate 07:08:42,414 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 07:08:42,414 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Follower): Received ElectionTimeout - switching to Candidate 07:08:42,415 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-oper_testRemoveShardReplica, member-2-shard-cars-oper_testRemoveShardReplica] 07:08:42,415 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Starting new election term 1 07:08:42,415 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:08:42,415 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:08:42,415 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:42,415 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from Follower to Candidate 07:08:42,415 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from Follower to Candidate 07:08:42,455 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 07:08:42,455 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Follower): Received ElectionTimeout - switching to Candidate 07:08:42,455 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-oper_testRemoveShardReplica, member-3-shard-people-oper_testRemoveShardReplica] 07:08:42,455 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Starting new election term 1 07:08:42,455 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:08:42,455 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:08:42,455 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:42,455 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from Follower to Candidate 07:08:42,456 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from Follower to Candidate 07:08:42,724 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Received ElectionTimeout 07:08:42,725 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Starting new election term 2 07:08:42,755 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Received ElectionTimeout 07:08:42,755 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Starting new election term 2 07:08:42,804 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Received ElectionTimeout 07:08:42,804 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Starting new election term 2 07:08:42,825 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Received ElectionTimeout 07:08:42,825 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Starting new election term 2 07:08:42,946 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:08:42,946 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 07:08:42,946 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 07:08:42,946 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 07:08:42,946 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 07:08:42,946 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:42,946 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 07:08:42,947 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 07:08:42,947 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 07:08:42,947 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 07:08:42,947 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 07:08:42,947 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 07:08:42,947 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 07:08:42,947 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 07:08:42,947 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:08:42,947 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 07:08:42,947 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 07:08:42,947 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 07:08:42,947 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 07:08:42,947 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 07:08:42,947 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:42,947 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 07:08:42,947 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 07:08:42,947 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 07:08:42,947 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 07:08:42,947 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 07:08:42,948 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 07:08:42,948 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 07:08:42,948 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 07:08:42,948 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-2-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 07:08:42,958 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:08:42,958 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 07:08:42,958 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 07:08:42,958 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 07:08:42,958 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 07:08:42,958 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:42,958 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 07:08:42,958 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:08:42,959 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 07:08:42,959 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 07:08:42,959 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 07:08:42,959 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 07:08:42,959 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 07:08:42,959 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 07:08:42,959 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 07:08:42,959 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 07:08:42,959 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:42,959 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 07:08:42,959 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 07:08:42,959 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 07:08:42,959 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 07:08:42,959 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 07:08:42,959 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 07:08:42,959 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 07:08:43,98 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:08:43,98 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 07:08:43,98 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 07:08:43,98 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 07:08:43,98 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 07:08:43,98 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:43,98 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 07:08:43,98 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 07:08:43,98 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 07:08:43,98 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 07:08:43,98 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 07:08:43,98 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-2-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 07:08:43,98 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:08:43,98 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 07:08:43,98 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 07:08:43,98 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 07:08:43,98 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 07:08:43,98 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:43,98 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 07:08:43,98 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 07:08:43,98 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 07:08:43,98 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 07:08:43,98 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 07:08:43,98 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 07:08:43,124 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Received ElectionTimeout 07:08:43,124 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Received ElectionTimeout 07:08:43,125 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Starting new election term 3 07:08:43,125 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Starting new election term 3 07:08:43,125 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1} to peer member-3-shard-cars-config_testRemoveShardReplica 07:08:43,125 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1} to peer member-2-shard-people-config_testRemoveShardReplica 07:08:43,125 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1} to peer member-2-shard-cars-config_testRemoveShardReplica 07:08:43,125 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1} to peer member-3-shard-people-config_testRemoveShardReplica 07:08:43,128 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:08:43,128 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica (Follower): Cluster state: [] 07:08:43,128 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:08:43,128 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:08:43,128 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:08:43,128 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Cluster state: [] 07:08:43,128 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:08:43,128 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:08:43,128 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:08:43,128 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:08:43,129 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:08:43,129 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:08:43,130 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:08:43,130 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica (Follower): Cluster state: [] 07:08:43,130 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:08:43,130 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:08:43,130 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:08:43,130 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:08:43,130 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:08:43,130 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica (Follower): Cluster state: [] 07:08:43,130 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:08:43,130 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-people-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:08:43,130 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:08:43,130 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:08:43,133 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): RaftRPC message received RequestVoteReply{term=3, voteGranted=true}, my term is 3 07:08:43,133 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): handleRequestVoteReply: RequestVoteReply{term=3, voteGranted=true}, current voteCount: 1 07:08:43,133 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): RaftRPC message received RequestVoteReply{term=3, voteGranted=true}, my term is 3 07:08:43,133 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): handleRequestVoteReply: RequestVoteReply{term=3, voteGranted=true}, current voteCount: 1 07:08:43,133 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-3-shard-cars-config_testRemoveShardReplica, member-2-shard-cars-config_testRemoveShardReplica] 07:08:43,133 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-2-shard-people-config_testRemoveShardReplica, member-3-shard-people-config_testRemoveShardReplica] 07:08:43,133 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:43,133 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:43,133 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:43,133 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:43,133 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 07:08:43,133 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 07:08:43,133 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 07:08:43,133 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 07:08:43,133 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: became leader with frontend state for [] 07:08:43,134 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: became leader with frontend state for [] 07:08:43,134 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from Candidate to Leader 07:08:43,134 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from Candidate to Leader 07:08:43,134 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testRemoveShardReplica, leaderId=member-1-shard-people-config_testRemoveShardReplica, leaderPayloadVersion=13] 07:08:43,134 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from Candidate to Leader 07:08:43,134 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testRemoveShardReplica, leaderId=member-1-shard-cars-config_testRemoveShardReplica, leaderPayloadVersion=13] 07:08:43,134 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from Candidate to Leader 07:08:43,134 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardReplica is ready 07:08:43,134 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} is now ready 07:08:43,134 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} 07:08:43,135 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardReplica status sync done false 07:08:43,135 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testRemoveShardReplica, leaderId=member-1-shard-people-config_testRemoveShardReplica, leaderPayloadVersion=13] 07:08:43,135 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardReplica, leaderId=member-1-shard-cars-config_testRemoveShardReplica, leaderPayloadVersion=13] 07:08:43,135 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardReplica is ready 07:08:43,135 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done false 07:08:43,136 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardReplica status sync done false 07:08:43,136 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveShardReplica status sync done false 07:08:43,136 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveShardReplica, leaderId=member-1-shard-cars-config_testRemoveShardReplica, leaderPayloadVersion=13] 07:08:43,136 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testRemoveShardReplica, leaderId=member-1-shard-people-config_testRemoveShardReplica, leaderPayloadVersion=13] 07:08:43,136 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardReplica is ready 07:08:43,136 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} is now ready 07:08:43,136 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-978735482] 07:08:43,137 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:08:43,137 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#2020213724] 07:08:43,137 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#2020213724] 07:08:43,137 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 07:08:43,137 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 07:08:43,137 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:43,137 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:43,138 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#1007391554] 07:08:43,138 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 07:08:43,138 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 07:08:43,138 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 07:08:43,138 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:43,138 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:43,138 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-920540737] 07:08:43,138 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-920540737] 07:08:43,138 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1158076466] 07:08:43,139 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 07:08:43,139 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#1118202765] 07:08:43,139 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#1118202765] 07:08:43,141 AM [main] [INFO] ClusterAdminRpcService - Removing replica for shard cars memberName member-3, datastoreType Config 07:08:43,142 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-3}] 07:08:43,144 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 07:08:43,144 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica, primaryVersion=13] 07:08:43,144 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica for shard member-3-shard-cars-config_testRemoveShardReplica 07:08:43,147 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: onRemoveServer: RemoveServer{serverId='member-3-shard-cars-config_testRemoveShardReplica'}, state: Idle 07:08:43,149 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]] 07:08:43,149 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 07:08:43,149 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 07:08:43,149 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=bfc1a6d7-ff63-4f47-af28-bed9511e7577}, logIndex: 0, isSendImmediate: true 07:08:43,149 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:43,149 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testRemoveShardReplica 07:08:43,149 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testRemoveShardReplica: AppendEntries{term=3, leaderId=member-1-shard-cars-config_testRemoveShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]]]} 07:08:43,149 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: Returning OK for operation RemoveServer{serverId='member-3-shard-cars-config_testRemoveShardReplica'} 07:08:43,151 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 07:08:43,151 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=member-1-shard-cars-config_testRemoveShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]]]} 07:08:43,151 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Number of entries to be appended = 1 07:08:43,151 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 07:08:43,151 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]] 07:08:43,151 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 07:08:43,151 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testRemoveShardReplica, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica, votingState=VOTING]] 07:08:43,151 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Log size is now 1 07:08:43,151 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:43,152 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 07:08:43,152 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done true 07:08:43,152 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveShardReplica} for shard cars 07:08:43,152 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard cars 07:08:43,152 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testRemoveShardReplica updated: matchIndex: 0, nextIndex: 1 07:08:43,153 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 07:08:43,153 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Setting last applied to 0 07:08:43,153 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Moving last applied index from -1 to 0 07:08:43,153 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ClusterAdminRpcService - Successfully removed replica for shard cars 07:08:43,153 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]] 07:08:43,153 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:43,153 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]] has been successfully replicated to a majority of followers 07:08:43,153 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Persisting ApplyJournalEntries with index=0 07:08:43,154 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-978735482] 07:08:43,154 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:08:43,154 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#2020213724] 07:08:43,154 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#2020213724] 07:08:43,154 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#1007391554] 07:08:43,154 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 07:08:43,154 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Commit index set to 0 07:08:43,154 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-920540737] 07:08:43,154 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-920540737] 07:08:43,154 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 07:08:43,155 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Setting last applied to 0 07:08:43,155 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Moving last applied index from -1 to 0 07:08:43,155 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]] 07:08:43,155 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Persisting ApplyJournalEntries with index=0 07:08:43,155 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1158076466] 07:08:43,155 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 07:08:43,155 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#1118202765] 07:08:43,158 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#1118202765] with 27000 ms timeout 07:08:43,159 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard cars has been removed 07:08:43,159 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [people] 07:08:43,159 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: onShutDown 07:08:43,159 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 07:08:43,160 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Successfully deleted prior snapshots 07:08:43,160 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testRemoveShardReplica 07:08:43,160 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#1118202765] 07:08:43,197 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 07:08:43,204 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Received ElectionTimeout 07:08:43,204 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Starting new election term 3 07:08:43,204 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1} to peer member-3-shard-cars-oper_testRemoveShardReplica 07:08:43,205 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1} to peer member-2-shard-cars-oper_testRemoveShardReplica 07:08:43,205 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1158076466] 07:08:43,205 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 07:08:43,205 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard cars not found - shards present: [people] 07:08:43,206 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=0} 07:08:43,206 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:08:43,206 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica (Follower): Cluster state: [] 07:08:43,206 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:08:43,206 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:08:43,206 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testRemoveShardReplica 07:08:43,206 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:08:43,207 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:08:43,207 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveShardReplica 07:08:43,207 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 07:08:43,207 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onShutDown 07:08:43,207 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onShutDown 07:08:43,207 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:08:43,207 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica (Follower): Cluster state: [] 07:08:43,207 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:08:43,207 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:08:43,207 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:08:43,207 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node [pekko://cluster-test@127.0.0.1:2559] as [Down] 07:08:43,207 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:08:43,207 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testRemoveShardReplica 07:08:43,208 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Stopping Shard member-2-shard-people-config_testRemoveShardReplica 07:08:43,208 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 07:08:43,208 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 07:08:43,208 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=0} 07:08:43,208 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveShardReplica 07:08:43,208 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveShardReplica 07:08:43,208 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 07:08:43,208 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onShutDown 07:08:43,208 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onShutDown 07:08:43,209 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testRemoveShardReplica 07:08:43,209 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testRemoveShardReplica 07:08:43,209 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 07:08:43,209 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 07:08:43,210 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:08:43,210 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:08:43,211 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): RaftRPC message received RequestVoteReply{term=3, voteGranted=true}, my term is 3 07:08:43,211 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): handleRequestVoteReply: RequestVoteReply{term=3, voteGranted=true}, current voteCount: 1 07:08:43,212 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardReplica, member-2-shard-cars-oper_testRemoveShardReplica] 07:08:43,212 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:43,212 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:43,212 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 07:08:43,212 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 07:08:43,212 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: became leader with frontend state for [] 07:08:43,212 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from Candidate to Leader 07:08:43,212 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testRemoveShardReplica, leaderId=member-1-shard-cars-oper_testRemoveShardReplica, leaderPayloadVersion=13] 07:08:43,212 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from Candidate to Leader 07:08:43,214 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveShardReplica status sync done false 07:08:43,214 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testRemoveShardReplica, leaderId=member-1-shard-cars-oper_testRemoveShardReplica, leaderPayloadVersion=13] 07:08:43,215 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Received ElectionTimeout 07:08:43,215 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Starting new election term 3 07:08:43,215 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1} to peer member-2-shard-people-oper_testRemoveShardReplica 07:08:43,215 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1} to peer member-3-shard-people-oper_testRemoveShardReplica 07:08:43,217 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:08:43,217 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica (Follower): Cluster state: [] 07:08:43,217 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:08:43,217 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-people-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:08:43,217 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:08:43,217 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:08:43,218 AM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Received gossip where this member has been downed, from [pekko://cluster-test@127.0.0.1:2558] 07:08:43,218 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Node has been marked as DOWN. Shutting down myself 07:08:43,218 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Shutting down... 07:08:43,218 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Successfully shut down 07:08:43,219 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 07:08:43,219 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): RaftRPC message received RequestVoteReply{term=3, voteGranted=true}, my term is 3 07:08:43,219 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): handleRequestVoteReply: RequestVoteReply{term=3, voteGranted=true}, current voteCount: 1 07:08:43,219 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:43,219 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-2-shard-people-oper_testRemoveShardReplica, member-3-shard-people-oper_testRemoveShardReplica] 07:08:43,219 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:43,219 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:43,219 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 07:08:43,219 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 07:08:43,219 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: became leader with frontend state for [] 07:08:43,220 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from Candidate to Leader 07:08:43,220 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testRemoveShardReplica, leaderId=member-1-shard-people-oper_testRemoveShardReplica, leaderPayloadVersion=13] 07:08:43,220 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from Candidate to Leader 07:08:43,220 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardReplica is ready 07:08:43,221 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveShardReplica status sync done false 07:08:43,222 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testRemoveShardReplica, leaderId=member-1-shard-people-oper_testRemoveShardReplica, leaderPayloadVersion=13] 07:08:43,222 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testRemoveShardReplica is ready 07:08:43,223 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 07:08:43,223 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:43,225 AM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:08:43,225 AM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:08:43,225 AM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:08:43,226 AM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:08:43,228 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 07:08:43,254 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:08:43,266 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [401471745040443989] 07:08:43,267 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 07:08:43,269 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Registered cluster JMX MBean [pekko:type=Cluster] 07:08:43,269 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 07:08:43,271 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:08:43,271 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:08:43,283 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:43,285 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:43,289 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 07:08:43,294 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 07:08:43,295 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#927681015] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:43,302 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 07:08:43,302 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveShardReplica 07:08:43,303 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveShardReplica 07:08:43,309 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@2dc6b182} 07:08:43,310 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} started 07:08:43,310 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:43,311 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:43,311 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveShardReplica 07:08:43,311 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveShardReplica 07:08:43,311 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:43,312 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-2-shard-cars-config_testRemoveShardReplica, persistent : true 07:08:43,313 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testRemoveShardReplica with journal batch size 1 07:08:43,313 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-2-shard-people-config_testRemoveShardReplica, persistent : true 07:08:43,313 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-2-shard-people-config_testRemoveShardReplica with journal batch size 1 07:08:43,314 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 07:08:43,315 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-232814590] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:43,315 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica#-277286894] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:43,315 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/shard-member-2:snapshot-read#-1809935025] 07:08:43,315 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#296670670] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:43,315 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#2139596325 created and ready for shard:member-2-shard-people-config_testRemoveShardReplica 07:08:43,315 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/shard-member-2:snapshot-read#112995999] 07:08:43,316 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 07:08:43,318 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 07:08:43,318 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveShardReplica 07:08:43,319 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#296670670 created and ready for shard:member-2-shard-cars-config_testRemoveShardReplica 07:08:43,319 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:08:43,319 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:08:43,319 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:08:43,319 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:08:43,319 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:08:43,319 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:43,319 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Received ReplicatedLogEntry for recovery: index: 0, size: 221 07:08:43,319 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:43,319 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testRemoveShardReplica, address=null, votingState=VOTING]] 07:08:43,319 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[pekko://cluster-test/deadLetters] 07:08:43,319 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 07:08:43,319 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveShardReplica is initialized 07:08:43,319 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Received apply journal entries for recovery, applying to state: 0 to 0 07:08:43,320 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Moving last applied index from -1 to 0 07:08:43,320 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:43,320 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica: Recovery completed in 58.49 μs - Switching actor to Follower - last log index = 0, last log term = 3, snapshot index = -1, snapshot term = -1, journal size = 1 07:08:43,320 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:08:43,320 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:43,320 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 07:08:43,320 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 07:08:43,320 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveShardReplica is initialized 07:08:43,321 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 07:08:43,321 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardReplica from null to Follower 07:08:43,321 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveShardReplica 07:08:43,322 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:08:43,322 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:43,323 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , received role change from null to Follower 07:08:43,323 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 07:08:43,323 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveShardReplica from null to Follower 07:08:43,323 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@16adc502} 07:08:43,323 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/shard-member-2:snapshot-read#-1809935025] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:43,323 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=1} started 07:08:43,323 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#2139596325] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:43,323 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/shard-member-2:snapshot-read#112995999] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:43,323 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#1145515044] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:43,323 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:43,323 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} 07:08:43,323 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveShardReplica 07:08:43,324 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveShardReplica 07:08:43,324 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-2-shard-cars-oper_testRemoveShardReplica, persistent : true 07:08:43,324 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Shard created : member-2-shard-people-oper_testRemoveShardReplica, persistent : true 07:08:43,325 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica-notifier#793022867 created and ready for shard:member-2-shard-people-oper_testRemoveShardReplica 07:08:43,325 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testRemoveShardReplica with journal batch size 1 07:08:43,325 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/shard-member-2:snapshot-read#338214768] 07:08:43,325 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica#-136966690] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:43,325 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica-notifier#1542052556 created and ready for shard:member-2-shard-cars-oper_testRemoveShardReplica 07:08:43,325 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica#-1155389625] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:43,325 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testRemoveShardReplica with journal batch size 1 07:08:43,327 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/shard-member-2:snapshot-read#-2050656766] 07:08:43,333 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:43,334 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:43,334 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 07:08:43,334 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveShardReplica is initialized 07:08:43,334 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:08:43,334 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:43,334 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 07:08:43,327 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:08:43,334 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 07:08:43,334 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:08:43,334 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:43,334 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:43,334 AM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:08:43,334 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:43,334 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveShardReplica from null to Follower 07:08:43,334 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 07:08:43,334 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveShardReplica is initialized 07:08:43,334 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 07:08:43,334 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 07:08:43,334 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveShardReplica from null to Follower 07:08:43,448 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveShardReplica status sync done true 07:08:43,450 AM [cluster-test-shard-dispatcher-21] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 311, lastApplied : -1, commitIndex : -1 07:08:43,527 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveShardReplica status sync done true 07:08:43,528 AM [cluster-test-shard-dispatcher-21] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 308, lastApplied : -1, commitIndex : -1 07:08:43,536 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveShardReplica status sync done true 07:08:43,537 AM [cluster-test-shard-dispatcher-21] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 313, lastApplied : -1, commitIndex : -1 07:08:43,770 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:43,848 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:43,857 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:43,965 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 07:08:44,89 AM [cluster-test-shard-dispatcher-21] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:44,169 AM [cluster-test-shard-dispatcher-21] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:44,177 AM [cluster-test-shard-dispatcher-21] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:44,227 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [5504065899577362811] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 07:08:44,227 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2559] ignored due to non-matching UID, quarantine requested for [5504065899577362811] but current is [401471745040443989]. Reason: ActorSystem terminated 07:08:44,277 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2558/system/clusterReceptionist/replicator#-329929556]] intended for system uid [5504065899577362811], self uid is [401471745040443989] 07:08:44,278 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardReplica status sync done false 07:08:44,278 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-1-shard-people-oper_testRemoveShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 0 - updating term 07:08:44,278 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardReplica status sync done true 07:08:44,278 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testRemoveShardReplica, leaderId=member-1-shard-cars-oper_testRemoveShardReplica, leaderPayloadVersion=13] 07:08:44,278 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testRemoveShardReplica, leaderId=member-1-shard-people-oper_testRemoveShardReplica, leaderPayloadVersion=13] 07:08:44,278 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done false 07:08:44,278 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done true 07:08:44,278 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardReplica status sync done false 07:08:44,278 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardReplica status sync done true 07:08:44,279 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardReplica, leaderId=member-1-shard-cars-config_testRemoveShardReplica, leaderPayloadVersion=13] 07:08:44,279 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testRemoveShardReplica, leaderId=member-1-shard-people-config_testRemoveShardReplica, leaderPayloadVersion=13] 07:08:44,279 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardReplica status sync done false 07:08:44,280 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardReplica status sync done true 07:08:44,281 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 07:08:44,281 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 07:08:44,282 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:44,282 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 07:08:44,282 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:44,282 AM [cluster-test-shard-dispatcher-21] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 1145, lastApplied : -1, commitIndex : -1 07:08:44,282 AM [cluster-test-shard-dispatcher-21] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 1126, lastApplied : 0, commitIndex : 0 07:08:44,284 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 07:08:44,284 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - New leader address: pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 07:08:44,284 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 07:08:44,408 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 07:08:44,489 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:44,497 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:44,728 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - New leader address: pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 07:08:44,728 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 07:08:44,729 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 444, lastApplied : 0, commitIndex : 0 07:08:44,729 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 07:08:44,730 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 447, lastApplied : -1, commitIndex : -1 07:08:44,808 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - New leader address: pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 07:08:44,808 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 07:08:44,809 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:44,809 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 526, lastApplied : -1, commitIndex : -1 07:08:44,816 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - New leader address: pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 07:08:44,816 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 07:08:44,817 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:44,817 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 534, lastApplied : -1, commitIndex : -1 07:08:45,48 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 07:08:45,48 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 315, lastApplied : 0, commitIndex : 0 07:08:45,49 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 07:08:45,127 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 07:08:45,128 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 07:08:45,137 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:45,137 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:45,303 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 07:08:45,369 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:45,369 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 07:08:45,369 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:45,449 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 07:08:45,449 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 07:08:45,457 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:45,457 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:45,689 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 07:08:45,689 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:45,689 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:45,769 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:45,770 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 07:08:45,777 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:45,777 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:46,9 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:46,9 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 312, lastApplied : 0, commitIndex : 0 07:08:46,9 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:46,87 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 07:08:46,88 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 07:08:46,97 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:46,98 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:46,325 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 07:08:46,328 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : 0, commitIndex : 0 07:08:46,328 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 07:08:46,328 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 07:08:46,409 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:46,409 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 07:08:46,417 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:46,417 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:46,647 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:46,648 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : 0, commitIndex : 0 07:08:46,649 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:46,729 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:46,729 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:46,737 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:46,737 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:46,969 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 07:08:46,969 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:46,970 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 07:08:47,49 AM [cluster-test-shard-dispatcher-26] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:47,50 AM [cluster-test-shard-dispatcher-26] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:47,57 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:47,57 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:47,287 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 07:08:47,288 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : 0, commitIndex : 0 07:08:47,288 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:47,344 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 07:08:47,368 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 07:08:47,369 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:47,377 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:47,377 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:47,609 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 07:08:47,610 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 07:08:47,610 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : 0, commitIndex : 0 07:08:47,687 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 07:08:47,688 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:47,697 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:47,697 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:47,929 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:47,930 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 07:08:47,930 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:48,8 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2558/system/clusterReceptionist/replicator#-329929556]] intended for system uid [5504065899577362811], self uid is [401471745040443989] 07:08:48,8 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:48,9 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 07:08:48,17 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:48,17 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:48,245 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica: At least 1 followers need to be active, Switching member-1-shard-cars-config_testRemoveShardReplica from Leader to IsolatedLeader 07:08:48,246 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica: At least 1 followers need to be active, Switching member-1-shard-people-config_testRemoveShardReplica from Leader to IsolatedLeader 07:08:48,246 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (IsolatedLeader): Election: Leader has following peers: [member-2-shard-cars-config_testRemoveShardReplica] 07:08:48,246 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardReplica: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:48,246 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (IsolatedLeader): Election: Leader has following peers: [member-2-shard-people-config_testRemoveShardReplica, member-3-shard-people-config_testRemoveShardReplica] 07:08:48,246 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:48,246 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:48,246 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 07:08:48,246 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 07:08:48,246 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 07:08:48,246 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 07:08:48,246 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from Leader to IsolatedLeader 07:08:48,246 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from Leader to IsolatedLeader 07:08:48,247 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from Leader to IsolatedLeader 07:08:48,247 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from Leader to IsolatedLeader 07:08:48,248 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2557/system/clusterReceptionist/replicator#-1320144535]] intended for system uid [5504065899577362811], self uid is [401471745040443989] 07:08:48,249 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:48,250 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - IsolatedLeader member-1-shard-people-config_testRemoveShardReplica switching from IsolatedLeader to Leader 07:08:48,250 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-2-shard-people-config_testRemoveShardReplica, member-3-shard-people-config_testRemoveShardReplica] 07:08:48,250 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:48,250 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:48,250 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 07:08:48,250 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 07:08:48,250 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from IsolatedLeader to Leader 07:08:48,250 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from IsolatedLeader to Leader 07:08:48,253 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : 0, commitIndex : 0 07:08:48,253 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - IsolatedLeader member-1-shard-cars-config_testRemoveShardReplica switching from IsolatedLeader to Leader 07:08:48,253 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testRemoveShardReplica] 07:08:48,253 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:48,253 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 323, lastApplied : -1, commitIndex : -1 07:08:48,254 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 07:08:48,254 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 07:08:48,254 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from IsolatedLeader to Leader 07:08:48,254 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from IsolatedLeader to Leader 07:08:48,254 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardReplica is ready 07:08:48,325 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica: At least 1 followers need to be active, Switching member-1-shard-cars-oper_testRemoveShardReplica from Leader to IsolatedLeader 07:08:48,325 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardReplica, member-2-shard-cars-oper_testRemoveShardReplica] 07:08:48,325 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:48,326 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:48,326 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 07:08:48,326 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 07:08:48,326 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from Leader to IsolatedLeader 07:08:48,326 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from Leader to IsolatedLeader 07:08:48,329 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:48,329 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - IsolatedLeader member-1-shard-cars-oper_testRemoveShardReplica switching from IsolatedLeader to Leader 07:08:48,329 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardReplica, member-2-shard-cars-oper_testRemoveShardReplica] 07:08:48,329 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:48,329 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:48,329 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 07:08:48,329 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 07:08:48,330 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from IsolatedLeader to Leader 07:08:48,330 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 07:08:48,330 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from IsolatedLeader to Leader 07:08:48,330 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardReplica is ready 07:08:48,335 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica: At least 1 followers need to be active, Switching member-1-shard-people-oper_testRemoveShardReplica from Leader to IsolatedLeader 07:08:48,335 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader): Election: Leader has following peers: [member-2-shard-people-oper_testRemoveShardReplica, member-3-shard-people-oper_testRemoveShardReplica] 07:08:48,335 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:48,335 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:48,335 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 07:08:48,335 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 07:08:48,335 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from Leader to IsolatedLeader 07:08:48,336 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from Leader to IsolatedLeader 07:08:48,336 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 07:08:48,336 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - IsolatedLeader member-1-shard-people-oper_testRemoveShardReplica switching from IsolatedLeader to Leader 07:08:48,336 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-2-shard-people-oper_testRemoveShardReplica, member-3-shard-people-oper_testRemoveShardReplica] 07:08:48,337 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:48,337 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:48,337 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 07:08:48,337 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 07:08:48,337 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from IsolatedLeader to Leader 07:08:48,337 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:48,337 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from IsolatedLeader to Leader 07:08:48,337 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardReplica is ready 07:08:48,362 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 07:08:48,569 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 315, lastApplied : -1, commitIndex : -1 07:08:48,569 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 314, lastApplied : 0, commitIndex : 0 07:08:48,570 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 315, lastApplied : -1, commitIndex : -1 07:08:48,649 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 07:08:48,649 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 07:08:48,657 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 07:08:48,657 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 07:08:48,889 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:48,889 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 07:08:48,889 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:48,968 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:48,969 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:48,976 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:48,976 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:49,66 AM [cluster-test-pekko.actor.default-dispatcher-18] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2559, Down)]. 07:08:49,66 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:49,66 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:49,208 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 07:08:49,209 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:49,210 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 07:08:49,288 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:49,288 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:49,297 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:49,298 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 07:08:49,382 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 07:08:49,441 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:49,441 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:49,528 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:49,528 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : 0, commitIndex : 0 07:08:49,529 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 07:08:49,608 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:49,608 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:49,617 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 07:08:49,618 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:49,849 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:49,849 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 07:08:49,849 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:49,928 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:49,929 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:49,937 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:49,937 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 07:08:50,26 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2558/system/clusterReceptionist/replicator#-329929556]] intended for system uid [5504065899577362811], self uid is [401471745040443989] 07:08:50,168 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:50,168 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 07:08:50,168 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:50,249 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:50,249 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:50,256 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:50,256 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:50,338 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Leader is removing unreachable node [pekko://cluster-test@127.0.0.1:2559] 07:08:50,340 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2559] ignored due to non-matching UID, quarantine requested for [5504065899577362811] but current is [401471745040443989]. Reason: Cluster member removed, previous status [Down] 07:08:50,340 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received MemberRemoved: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:50,340 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberRemoved: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:50,402 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 07:08:50,456 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2559] ignored due to non-matching UID, quarantine requested for [5504065899577362811] but current is [401471745040443989]. Reason: Cluster member removed, previous status [Down] 07:08:50,457 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberRemoved: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:50,457 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberRemoved: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:50,488 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:50,488 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:50,489 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 07:08:50,567 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 07:08:50,567 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 07:08:50,576 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:50,577 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:50,808 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:50,808 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:50,808 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 07:08:50,887 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:50,889 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 07:08:50,896 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:50,896 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:08:51,129 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 07:08:51,129 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:51,130 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 07:08:51,208 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 07:08:51,208 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:51,217 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:08:51,217 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 07:08:51,423 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 07:08:51,426 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 07:08:51,428 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:08:51,428 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:08:51,428 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 07:08:51,428 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 07:08:51,428 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 07:08:51,428 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 07:08:51,428 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 07:08:51,428 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 07:08:51,428 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 07:08:51,428 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 07:08:51,428 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardReplica is ready 07:08:51,428 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 07:08:51,428 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:51,428 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardReplica is ready 07:08:51,428 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 07:08:51,428 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 07:08:51,428 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 07:08:51,428 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 07:08:51,428 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:51,428 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 07:08:51,428 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 07:08:51,428 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 07:08:51,428 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 07:08:51,428 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 07:08:51,428 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 07:08:51,428 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 07:08:51,428 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardReplica is ready 07:08:51,428 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 07:08:51,428 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 07:08:51,428 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} is now ready 07:08:51,428 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardReplica is ready 07:08:51,428 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 07:08:51,428 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#927681015] 07:08:51,429 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 07:08:51,429 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-232814590] 07:08:51,429 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-232814590] 07:08:51,429 AM [main] [INFO] ClusterAdminRpcService - Removing replica for shard cars memberName member-2, datastoreType Config 07:08:51,429 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-1: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-2}] 07:08:51,430 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 07:08:51,430 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: LocalPrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#2020213724, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@e8ab191] 07:08:51,431 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-1: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#2020213724 for shard member-2-shard-cars-config_testRemoveShardReplica 07:08:51,431 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: onRemoveServer: RemoveServer{serverId='member-2-shard-cars-config_testRemoveShardReplica'}, state: Idle 07:08:51,432 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: New server configuration : [ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]] 07:08:51,432 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Persist data SimpleReplicatedLogEntry [index=1, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 07:08:51,432 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 07:08:51,432 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: Returning OK for operation RemoveServer{serverId='member-2-shard-cars-config_testRemoveShardReplica'} 07:08:51,432 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-1: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveShardReplica} for shard cars 07:08:51,432 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-1: Leader shard successfully removed the replica shard cars 07:08:51,432 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] ClusterAdminRpcService - Successfully removed replica for shard cars 07:08:51,432 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-978735482] 07:08:51,432 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:08:51,432 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#2020213724] 07:08:51,432 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#2020213724] 07:08:51,433 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#927681015] 07:08:51,433 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 07:08:51,433 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 07:08:51,433 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-232814590] 07:08:51,433 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Moving last applied index from 0 to 1 07:08:51,433 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]] 07:08:51,433 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]] has been successfully replicated to a majority of followers 07:08:51,433 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Persisting ApplyJournalEntries with index=1 07:08:51,434 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-232814590] with 27000 ms timeout 07:08:51,435 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2 : Local Shard replica for shard cars has been removed 07:08:51,435 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [people] 07:08:51,435 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onShutDown 07:08:51,435 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 07:08:51,435 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testRemoveShardReplica 07:08:51,435 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Successfully deleted prior snapshots 07:08:51,435 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-2 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-232814590] 07:08:51,448 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 07:08:51,448 AM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 07:08:51,483 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#927681015] 07:08:51,483 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 07:08:51,483 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Local shard cars not found - shards present: [people] 07:08:51,484 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} 07:08:51,484 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveShardReplica 07:08:51,484 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 1 shards 07:08:51,484 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onShutDown 07:08:51,485 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Stopping Shard member-2-shard-people-config_testRemoveShardReplica 07:08:51,485 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 07:08:51,485 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 07:08:51,485 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=1} 07:08:51,486 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveShardReplica 07:08:51,486 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveShardReplica 07:08:51,486 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 07:08:51,486 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onShutDown 07:08:51,486 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onShutDown 07:08:51,486 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testRemoveShardReplica 07:08:51,487 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testRemoveShardReplica 07:08:51,487 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 07:08:51,487 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 07:08:51,488 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:08:51,489 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:08:51,497 AM [cluster-test-pekko.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:08:51,498 AM [cluster-test-pekko.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:08:51,498 AM [cluster-test-pekko.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:08:51,498 AM [cluster-test-pekko.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:08:51,499 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 07:08:51,511 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} 07:08:51,512 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testRemoveShardReplica 07:08:51,512 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 07:08:51,512 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: onShutDown 07:08:51,512 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Stopping Shard member-3-shard-people-config_testRemoveShardReplica 07:08:51,513 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 07:08:51,513 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 07:08:51,513 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardReplica, generation=0} 07:08:51,514 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testRemoveShardReplica 07:08:51,514 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testRemoveShardReplica 07:08:51,514 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 07:08:51,514 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: onShutDown 07:08:51,514 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: onShutDown 07:08:51,514 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testRemoveShardReplica 07:08:51,515 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testRemoveShardReplica 07:08:51,515 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 07:08:51,515 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 07:08:51,517 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:08:51,517 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:08:52,377 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 07:08:52,505 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [401471745040443989] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 07:08:52,535 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-7387743005627041772] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 07:08:52,540 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:08:52,540 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:08:52,547 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [01/31/2025 07:08:52.549] [cluster-test-pekko.actor.internal-dispatcher-9] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:08:52,557 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} 07:08:52,557 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testRemoveShardReplica 07:08:52,558 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testRemoveShardReplica 07:08:52,558 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: onShutDown 07:08:52,558 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 07:08:52,558 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@477204af 07:08:52,558 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-config_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@477204af 07:08:52,558 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: onShutDown 07:08:52,558 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: Initiating leader transfer 07:08:52,558 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6939a0e6 07:08:52,558 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-config_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6939a0e6 07:08:52,558 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testRemoveShardReplica, leaderId=null, leaderPayloadVersion=13] 07:08:52,558 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testRemoveShardReplica 07:08:52,558 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveShardReplica: pauseLeader successfully completed - doing transfer 07:08:52,558 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Attempting to transfer leadership 07:08:52,558 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:52,559 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:53,45 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveShardReplica: leader transfer aborted 07:08:53,45 AM [cluster-test-shard-dispatcher-21] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveShardReplica: Failed to transfer leadership in 486.7 ms 07:08:53,45 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: In unpauseLeader 07:08:53,45 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: leader transfer failed - sending PoisonPill 07:08:53,45 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Leadership transfer expired 07:08:53,46 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - Stopping Shard member-1-shard-people-config_testRemoveShardReplica 07:08:53,46 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 07:08:53,47 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 07:08:53,47 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardReplica, generation=0} 07:08:53,48 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testRemoveShardReplica 07:08:53,48 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testRemoveShardReplica 07:08:53,48 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 07:08:53,48 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: onShutDown 07:08:53,49 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Initiating leader transfer 07:08:53,49 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: onShutDown 07:08:53,49 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: Initiating leader transfer 07:08:53,49 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testRemoveShardReplica, leaderId=null, leaderPayloadVersion=13] 07:08:53,49 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@52d4c5ee 07:08:53,49 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@52d4c5ee 07:08:53,49 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testRemoveShardReplica, leaderId=null, leaderPayloadVersion=13] 07:08:53,49 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveShardReplica: pauseLeader successfully completed - doing transfer 07:08:53,49 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Attempting to transfer leadership 07:08:53,49 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@39a554ec 07:08:53,49 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:53,49 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-oper_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@39a554ec 07:08:53,49 AM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveShardReplica: pauseLeader successfully completed - doing transfer 07:08:53,49 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Attempting to transfer leadership 07:08:53,49 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:53,49 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:53,49 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:53,446 AM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveShardReplica: leader transfer aborted 07:08:53,446 AM [cluster-test-shard-dispatcher-23] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveShardReplica: Failed to transfer leadership in 397.3 ms 07:08:53,446 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: In unpauseLeader 07:08:53,446 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: leader transfer failed - sending PoisonPill 07:08:53,446 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Leadership transfer expired 07:08:53,446 AM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica: At least 1 followers need to be active, Switching member-1-shard-cars-oper_testRemoveShardReplica from Leader to IsolatedLeader 07:08:53,447 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardReplica, member-2-shard-cars-oper_testRemoveShardReplica] 07:08:53,447 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:53,447 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:53,447 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 07:08:53,447 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 07:08:53,447 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from Leader to IsolatedLeader 07:08:53,447 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from Leader to IsolatedLeader 07:08:53,447 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testRemoveShardReplica 07:08:53,455 AM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveShardReplica: leader transfer aborted 07:08:53,455 AM [cluster-test-shard-dispatcher-23] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveShardReplica: Failed to transfer leadership in 406.0 ms 07:08:53,455 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: In unpauseLeader 07:08:53,455 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: leader transfer failed - sending PoisonPill 07:08:53,455 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Leadership transfer expired 07:08:53,455 AM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica: At least 1 followers need to be active, Switching member-1-shard-people-oper_testRemoveShardReplica from Leader to IsolatedLeader 07:08:53,455 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader): Election: Leader has following peers: [member-2-shard-people-oper_testRemoveShardReplica, member-3-shard-people-oper_testRemoveShardReplica] 07:08:53,456 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:53,456 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:53,456 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 07:08:53,456 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 07:08:53,456 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from Leader to IsolatedLeader 07:08:53,456 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from Leader to IsolatedLeader 07:08:53,456 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testRemoveShardReplica 07:08:53,457 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 07:08:53,457 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 07:08:53,458 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:08:53,458 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:08:53,462 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [01/31/2025 07:08:53.465] [cluster-test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:08:53,488 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 07:08:53,500 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-3097334140079229405] 07:08:53,500 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 07:08:53,503 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 07:08:53,503 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 07:08:53,505 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:08:53,505 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:08:53,518 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:53,519 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 07:08:53,519 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 07:08:53,520 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 07:08:53,520 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:53,523 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveAllShardReplicas 07:08:53,529 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 07:08:53,529 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#720286852] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:53,534 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 07:08:53,534 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testRemoveAllShardReplicas 07:08:53,535 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testRemoveAllShardReplicas 07:08:53,535 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:53,535 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 07:08:53,535 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 07:08:53,541 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 07:08:53,543 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@c2a4f14} 07:08:53,543 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} started 07:08:53,544 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:53,545 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:53,547 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:53,547 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testRemoveAllShardReplicas 07:08:53,547 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveAllShardReplicas 07:08:53,548 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 07:08:53,548 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 07:08:53,548 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas on shard member-1-shard-cars-config_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#1590949546] 07:08:53,548 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testRemoveAllShardReplicas 07:08:53,548 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 07:08:53,548 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-1-shard-cars-config_testRemoveAllShardReplicas, persistent : true 07:08:53,548 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 07:08:53,548 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas on shard member-1-shard-people-config_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#-755534205] 07:08:53,549 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - Shard created : member-1-shard-people-config_testRemoveAllShardReplicas, persistent : true 07:08:53,549 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testRemoveAllShardReplicas with journal batch size 1 07:08:53,549 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas-notifier#1361478137 created and ready for shard:member-1-shard-people-config_testRemoveAllShardReplicas 07:08:53,550 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas/shard-member-1:snapshot-read#1466123588] 07:08:53,550 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - Starting recovery for member-1-shard-people-config_testRemoveAllShardReplicas with journal batch size 1 07:08:53,550 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#1590949546] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:53,550 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#-755534205] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:53,550 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas-notifier#1361478137] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:53,550 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas-notifier#-478039623 created and ready for shard:member-1-shard-cars-config_testRemoveAllShardReplicas 07:08:53,550 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas/shard-member-1:snapshot-read#78857129] 07:08:53,550 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas/shard-member-1:snapshot-read#1466123588] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:53,551 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 07:08:53,551 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:53,551 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:53,551 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:53,551 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 07:08:53,551 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:53,551 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:53,551 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , received role change from null to Follower 07:08:53,551 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 07:08:53,551 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:53,552 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 07:08:53,552 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testRemoveAllShardReplicas is initialized 07:08:53,552 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 07:08:53,552 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testRemoveAllShardReplicas is initialized 07:08:53,552 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 07:08:53,552 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveAllShardReplicas from null to Follower 07:08:53,552 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 07:08:53,552 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testRemoveAllShardReplicas 07:08:53,552 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , received role change from null to Follower 07:08:53,552 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 07:08:53,552 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveAllShardReplicas from null to Follower 07:08:53,552 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testRemoveAllShardReplicas 07:08:53,552 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:53,552 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 07:08:53,552 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 07:08:53,555 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas-notifier#-478039623] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:53,555 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#1535813270] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:53,555 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas/shard-member-1:snapshot-read#78857129] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:53,558 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 07:08:53,559 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} started 07:08:53,559 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@3c3bb14e} 07:08:53,559 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:53,559 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testRemoveAllShardReplicas 07:08:53,560 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 07:08:53,560 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 07:08:53,560 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas on shard member-1-shard-cars-oper_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas#1070282189] 07:08:53,560 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testRemoveAllShardReplicas 07:08:53,561 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 07:08:53,561 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 07:08:53,561 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas on shard member-1-shard-people-oper_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas#-813830241] 07:08:53,562 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas#1070282189] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:53,562 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - Shard created : member-1-shard-cars-oper_testRemoveAllShardReplicas, persistent : true 07:08:53,562 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas-notifier#2078678746 created and ready for shard:member-1-shard-cars-oper_testRemoveAllShardReplicas 07:08:53,563 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas-notifier#2078678746] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:53,563 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas/shard-member-1:snapshot-read#49779077] 07:08:53,563 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Shard created : member-1-shard-people-oper_testRemoveAllShardReplicas, persistent : true 07:08:53,563 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas-notifier#1652361892 created and ready for shard:member-1-shard-people-oper_testRemoveAllShardReplicas 07:08:53,564 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testRemoveAllShardReplicas with journal batch size 1 07:08:53,564 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas/shard-member-1:snapshot-read#-1880847948] 07:08:53,566 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testRemoveAllShardReplicas with journal batch size 1 07:08:53,567 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:53,567 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:53,567 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 07:08:53,567 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testRemoveAllShardReplicas is initialized 07:08:53,567 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 07:08:53,567 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:53,567 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:53,567 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:53,567 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 07:08:53,567 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 07:08:53,567 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:53,567 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testRemoveAllShardReplicas is initialized 07:08:53,567 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , received role change from null to Follower 07:08:53,567 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , received role change from null to Follower 07:08:53,567 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 07:08:53,567 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveAllShardReplicas from null to Follower 07:08:53,568 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 07:08:53,568 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveAllShardReplicas from null to Follower 07:08:53,569 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:08:53,586 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-4700198422083289251] 07:08:53,587 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 07:08:53,590 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:08:53,590 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 07:08:53,591 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:08:53,591 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:08:53,611 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:53,614 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:53,621 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveAllShardReplicas 07:08:53,628 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 07:08:53,635 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1545984901] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:53,639 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 07:08:53,639 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveAllShardReplicas 07:08:53,640 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveAllShardReplicas 07:08:53,643 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 07:08:53,646 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@17f0bb05} 07:08:53,646 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} started 07:08:53,647 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:53,647 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:53,647 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveAllShardReplicas 07:08:53,649 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:53,651 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveAllShardReplicas 07:08:53,654 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveAllShardReplicas 07:08:53,655 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Shard created : member-2-shard-cars-config_testRemoveAllShardReplicas, persistent : true 07:08:53,655 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testRemoveAllShardReplicas with journal batch size 1 07:08:53,656 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Shard created : member-2-shard-people-config_testRemoveAllShardReplicas, persistent : true 07:08:53,656 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Starting recovery for member-2-shard-people-config_testRemoveAllShardReplicas with journal batch size 1 07:08:53,657 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 07:08:53,657 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#1787224382] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:53,657 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas-notifier#289587364 created and ready for shard:member-2-shard-cars-config_testRemoveAllShardReplicas 07:08:53,657 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas#-1565320357] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:53,657 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#-2013328199] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:53,658 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas/shard-member-2:snapshot-read#-1907887224] 07:08:53,658 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas-notifier#289587364] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:53,658 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas-notifier#-1069436974 created and ready for shard:member-2-shard-people-config_testRemoveAllShardReplicas 07:08:53,658 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas/shard-member-2:snapshot-read#12990200] 07:08:53,658 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas/shard-member-2:snapshot-read#-1907887224] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:53,660 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 07:08:53,664 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 07:08:53,664 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 07:08:53,666 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} started 07:08:53,670 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas-notifier#-1069436974] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:53,671 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas/shard-member-2:snapshot-read#12990200] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:53,671 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:53,671 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 07:08:53,671 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 07:08:53,671 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 07:08:53,671 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 07:08:53,674 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:53,674 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:53,674 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 07:08:53,674 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveAllShardReplicas is initialized 07:08:53,688 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 07:08:53,688 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveAllShardReplicas 07:08:53,689 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveAllShardReplicas 07:08:53,689 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:53,689 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 07:08:53,689 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 07:08:53,689 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@7e5fc394} 07:08:53,689 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:53,689 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveAllShardReplicas 07:08:53,690 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 07:08:53,690 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 07:08:53,690 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: updated peer member-1-shard-cars-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas on shard member-2-shard-cars-oper_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas#-1440596848] 07:08:53,690 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveAllShardReplicas 07:08:53,690 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 07:08:53,690 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 07:08:53,690 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: updated peer member-1-shard-people-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas on shard member-2-shard-people-oper_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas#652435388] 07:08:53,691 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - Shard created : member-2-shard-people-oper_testRemoveAllShardReplicas, persistent : true 07:08:53,692 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testRemoveAllShardReplicas with journal batch size 1 07:08:53,692 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:53,694 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:53,692 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas-notifier#-826663717 created and ready for shard:member-2-shard-people-oper_testRemoveAllShardReplicas 07:08:53,694 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:53,692 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas#652435388] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:53,692 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas/shard-member-2:snapshot-read#-1216384716] 07:08:53,694 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:53,694 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas-notifier#-826663717] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:53,694 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 07:08:53,694 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveAllShardReplicas is initialized 07:08:53,694 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 07:08:53,694 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveAllShardReplicas is initialized 07:08:53,694 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 07:08:53,694 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:53,694 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveAllShardReplicas , received role change from null to Follower 07:08:53,694 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 07:08:53,695 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 07:08:53,695 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveAllShardReplicas from null to Follower 07:08:53,695 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 07:08:53,695 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:53,695 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveAllShardReplicas , received role change from null to Follower 07:08:53,695 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 07:08:53,695 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 07:08:53,695 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveAllShardReplicas from null to Follower 07:08:53,695 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 07:08:53,695 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:53,696 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 07:08:53,696 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveAllShardReplicas , received role change from null to Follower 07:08:53,696 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 07:08:53,696 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveAllShardReplicas from null to Follower 07:08:53,696 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - Shard created : member-2-shard-cars-oper_testRemoveAllShardReplicas, persistent : true 07:08:53,696 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 07:08:53,697 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas-notifier#-1093584445 created and ready for shard:member-2-shard-cars-oper_testRemoveAllShardReplicas 07:08:53,699 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas/shard-member-2:snapshot-read#834790762] 07:08:53,701 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testRemoveAllShardReplicas with journal batch size 1 07:08:53,702 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:53,704 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:53,704 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 07:08:53,704 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:53,704 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveAllShardReplicas , received role change from null to Follower 07:08:53,705 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 07:08:53,705 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveAllShardReplicas is initialized 07:08:53,708 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 07:08:53,708 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 07:08:53,708 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [3765145098590628681] 07:08:53,708 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 07:08:53,711 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveAllShardReplicas from null to Follower 07:08:53,711 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:08:53,711 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 07:08:53,714 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:08:53,714 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:08:53,720 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:53,723 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:53,726 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveAllShardReplicas 07:08:53,733 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1410520283] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:53,734 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 07:08:53,737 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 07:08:53,737 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testRemoveAllShardReplicas 07:08:53,737 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testRemoveAllShardReplicas 07:08:53,739 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 07:08:53,740 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@1a63a513} 07:08:53,740 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} started 07:08:53,740 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:53,742 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:53,744 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:53,744 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testRemoveAllShardReplicas 07:08:53,745 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveAllShardReplicas 07:08:53,751 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testRemoveAllShardReplicas 07:08:53,751 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#1444518893] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:53,752 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-3-shard-cars-config_testRemoveAllShardReplicas, persistent : true 07:08:53,752 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#216494914] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:53,752 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas-notifier#-1564982062 created and ready for shard:member-3-shard-cars-config_testRemoveAllShardReplicas 07:08:53,752 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testRemoveAllShardReplicas with journal batch size 1 07:08:53,752 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas-notifier#-1564982062] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:53,753 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-3-shard-people-config_testRemoveAllShardReplicas, persistent : true 07:08:53,753 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-3-shard-people-config_testRemoveAllShardReplicas with journal batch size 1 07:08:53,754 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 07:08:53,764 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas/shard-member-3:snapshot-read#-787139021] 07:08:53,764 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas/shard-member-3:snapshot-read#-519285516] 07:08:53,764 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas#-727120548] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:53,766 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:53,766 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:53,766 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:53,766 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:53,766 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 07:08:53,766 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:53,766 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 07:08:53,766 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testRemoveAllShardReplicas is initialized 07:08:53,766 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 07:08:53,766 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testRemoveAllShardReplicas is initialized 07:08:53,766 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 07:08:53,768 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas/shard-member-3:snapshot-read#-787139021] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:53,768 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas/shard-member-3:snapshot-read#-519285516] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:53,768 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testRemoveAllShardReplicas 07:08:53,769 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 07:08:53,769 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:53,769 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveAllShardReplicas , received role change from null to Follower 07:08:53,769 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas-notifier#308006367 created and ready for shard:member-3-shard-people-config_testRemoveAllShardReplicas 07:08:53,769 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveAllShardReplicas , received role change from null to Follower 07:08:53,769 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 07:08:53,769 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testRemoveAllShardReplicas from null to Follower 07:08:53,769 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 07:08:53,769 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveAllShardReplicas from null to Follower 07:08:53,769 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas-notifier#308006367] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:53,770 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testRemoveAllShardReplicas 07:08:53,770 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 07:08:53,774 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@104f5b6d} 07:08:53,774 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} started 07:08:53,774 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 07:08:53,776 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:53,776 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testRemoveAllShardReplicas 07:08:53,777 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testRemoveAllShardReplicas 07:08:53,777 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Shard created : member-3-shard-cars-oper_testRemoveAllShardReplicas, persistent : true 07:08:53,778 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testRemoveAllShardReplicas with journal batch size 1 07:08:53,778 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Shard created : member-3-shard-people-oper_testRemoveAllShardReplicas, persistent : true 07:08:53,778 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testRemoveAllShardReplicas with journal batch size 1 07:08:53,780 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#2043634457] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:53,780 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#-386378040] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:53,780 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas-notifier#711898943 created and ready for shard:member-3-shard-cars-oper_testRemoveAllShardReplicas 07:08:53,780 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas/shard-member-3:snapshot-read#-1915331503] 07:08:53,780 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas-notifier#1825066798 created and ready for shard:member-3-shard-people-oper_testRemoveAllShardReplicas 07:08:53,780 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas/shard-member-3:snapshot-read#-1780405539] 07:08:53,781 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:53,782 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:53,782 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:53,782 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:53,782 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 07:08:53,783 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testRemoveAllShardReplicas is initialized 07:08:53,782 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 07:08:53,783 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:53,784 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 07:08:53,784 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testRemoveAllShardReplicas is initialized 07:08:53,784 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 07:08:53,784 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:53,784 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 07:08:53,784 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveAllShardReplicas , received role change from null to Follower 07:08:53,784 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 07:08:53,784 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveAllShardReplicas , received role change from null to Follower 07:08:53,784 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testRemoveAllShardReplicas from null to Follower 07:08:53,784 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 07:08:53,785 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testRemoveAllShardReplicas from null to Follower 07:08:53,787 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:53,787 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 07:08:53,787 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 07:08:53,787 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 07:08:53,787 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 07:08:53,787 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 07:08:53,787 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 07:08:53,787 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:53,787 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 07:08:53,787 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 07:08:53,787 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 07:08:53,787 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 07:08:53,787 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 07:08:53,787 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 07:08:53,787 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 07:08:53,874 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 07:08:53,874 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Follower): Received ElectionTimeout - switching to Candidate 07:08:53,874 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testRemoveAllShardReplicas, member-3-shard-cars-config_testRemoveAllShardReplicas] 07:08:53,874 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Starting new election term 1 07:08:53,874 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:08:53,874 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 07:08:53,874 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:53,874 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , received role change from Follower to Candidate 07:08:53,874 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveAllShardReplicas from Follower to Candidate 07:08:53,894 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 07:08:53,894 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Follower): Received ElectionTimeout - switching to Candidate 07:08:53,894 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-oper_testRemoveAllShardReplicas, member-3-shard-people-oper_testRemoveAllShardReplicas] 07:08:53,894 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 1 07:08:53,894 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:08:53,894 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 07:08:53,894 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:53,894 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , received role change from Follower to Candidate 07:08:53,894 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveAllShardReplicas from Follower to Candidate 07:08:53,924 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 07:08:53,924 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Follower): Received ElectionTimeout - switching to Candidate 07:08:53,924 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-oper_testRemoveAllShardReplicas, member-3-shard-cars-oper_testRemoveAllShardReplicas] 07:08:53,924 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 1 07:08:53,924 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:08:53,924 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 07:08:53,924 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:53,924 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , received role change from Follower to Candidate 07:08:53,924 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveAllShardReplicas from Follower to Candidate 07:08:53,954 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 07:08:53,954 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Follower): Received ElectionTimeout - switching to Candidate 07:08:53,954 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Election: Candidate has following voting peers: [member-3-shard-people-config_testRemoveAllShardReplicas, member-2-shard-people-config_testRemoveAllShardReplicas] 07:08:53,954 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Starting new election term 1 07:08:53,954 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:08:53,954 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 07:08:53,954 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:53,954 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , received role change from Follower to Candidate 07:08:53,954 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveAllShardReplicas from Follower to Candidate 07:08:54,294 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 07:08:54,294 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 07:08:54,294 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 07:08:54,294 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Starting new election term 2 07:08:54,294 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 2 07:08:54,294 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 2 07:08:54,304 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 07:08:54,304 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Starting new election term 2 07:08:54,525 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 07:08:54,525 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 07:08:54,526 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:08:54,526 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 07:08:54,526 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 07:08:54,526 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:08:54,526 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 07:08:54,526 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 07:08:54,526 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 07:08:54,526 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:54,526 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 07:08:54,526 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 07:08:54,526 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 07:08:54,526 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 07:08:54,526 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 07:08:54,526 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 07:08:54,526 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 07:08:54,526 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:54,526 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 07:08:54,526 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 07:08:54,526 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 07:08:54,526 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 07:08:54,526 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 07:08:54,526 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 07:08:54,526 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 07:08:54,526 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 07:08:54,526 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 07:08:54,526 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 07:08:54,526 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-2-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 07:08:54,526 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 07:08:54,537 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:08:54,537 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 07:08:54,537 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 07:08:54,537 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 07:08:54,537 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 07:08:54,537 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:54,537 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 07:08:54,537 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 07:08:54,537 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 07:08:54,537 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 07:08:54,537 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 07:08:54,537 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 07:08:54,537 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:08:54,537 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 07:08:54,537 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 07:08:54,537 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 07:08:54,537 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 07:08:54,537 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 07:08:54,537 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:54,537 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 07:08:54,537 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 07:08:54,537 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 07:08:54,537 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 07:08:54,537 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 07:08:54,624 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 07:08:54,624 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Starting new election term 3 07:08:54,624 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1} to peer member-2-shard-cars-config_testRemoveAllShardReplicas 07:08:54,624 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1} to peer member-3-shard-cars-config_testRemoveAllShardReplicas 07:08:54,624 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:08:54,624 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 07:08:54,624 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 07:08:54,624 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 07:08:54,624 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 07:08:54,624 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:54,624 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 07:08:54,625 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 07:08:54,625 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 07:08:54,625 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 07:08:54,625 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 07:08:54,625 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 07:08:54,625 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:08:54,625 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 07:08:54,625 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 07:08:54,625 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 07:08:54,625 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 07:08:54,625 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:54,625 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 07:08:54,625 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 07:08:54,625 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 07:08:54,625 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 07:08:54,625 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 07:08:54,625 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - Peer address for peer member-2-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 07:08:54,626 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:08:54,626 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): Cluster state: [] 07:08:54,626 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:08:54,626 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:08:54,626 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:08:54,626 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:08:54,626 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:08:54,626 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Cluster state: [] 07:08:54,626 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:08:54,626 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:08:54,626 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:08:54,626 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:08:54,628 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): RaftRPC message received RequestVoteReply{term=3, voteGranted=true}, my term is 3 07:08:54,628 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): handleRequestVoteReply: RequestVoteReply{term=3, voteGranted=true}, current voteCount: 1 07:08:54,628 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testRemoveAllShardReplicas, member-3-shard-cars-config_testRemoveAllShardReplicas] 07:08:54,628 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:54,628 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:54,628 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 07:08:54,628 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 07:08:54,628 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: became leader with frontend state for [] 07:08:54,628 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , received role change from Candidate to Leader 07:08:54,628 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testRemoveAllShardReplicas, leaderId=member-1-shard-cars-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 07:08:54,628 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveAllShardReplicas from Candidate to Leader 07:08:54,630 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveAllShardReplicas, leaderId=member-1-shard-cars-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 07:08:54,630 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveAllShardReplicas status sync done false 07:08:54,630 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveAllShardReplicas status sync done false 07:08:54,630 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveAllShardReplicas, leaderId=member-1-shard-cars-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 07:08:54,630 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 07:08:54,630 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:54,630 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 07:08:54,630 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:54,634 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 07:08:54,634 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 3 07:08:54,634 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1} to peer member-2-shard-people-oper_testRemoveAllShardReplicas 07:08:54,634 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1} to peer member-3-shard-people-oper_testRemoveAllShardReplicas 07:08:54,635 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:08:54,635 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Cluster state: [] 07:08:54,635 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:08:54,635 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:08:54,635 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:08:54,635 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:08:54,635 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:08:54,635 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): Cluster state: [] 07:08:54,635 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:08:54,635 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:08:54,635 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:08:54,635 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:08:54,636 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): RaftRPC message received RequestVoteReply{term=3, voteGranted=true}, my term is 3 07:08:54,636 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): handleRequestVoteReply: RequestVoteReply{term=3, voteGranted=true}, current voteCount: 1 07:08:54,636 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Election: Leader has following peers: [member-2-shard-people-oper_testRemoveAllShardReplicas, member-3-shard-people-oper_testRemoveAllShardReplicas] 07:08:54,636 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:54,636 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:54,636 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 07:08:54,636 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 07:08:54,637 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: became leader with frontend state for [] 07:08:54,637 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testRemoveAllShardReplicas, leaderId=member-1-shard-people-oper_testRemoveAllShardReplicas, leaderPayloadVersion=13] 07:08:54,637 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , received role change from Candidate to Leader 07:08:54,637 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveAllShardReplicas from Candidate to Leader 07:08:54,637 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveAllShardReplicas status sync done false 07:08:54,637 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveAllShardReplicas status sync done false 07:08:54,637 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testRemoveAllShardReplicas, leaderId=member-1-shard-people-oper_testRemoveAllShardReplicas, leaderPayloadVersion=13] 07:08:54,638 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 07:08:54,638 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:54,638 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testRemoveAllShardReplicas, leaderId=member-1-shard-people-oper_testRemoveAllShardReplicas, leaderPayloadVersion=13] 07:08:54,638 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 07:08:54,638 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:54,674 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 07:08:54,674 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 3 07:08:54,674 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1} to peer member-2-shard-cars-oper_testRemoveAllShardReplicas 07:08:54,674 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1} to peer member-3-shard-cars-oper_testRemoveAllShardReplicas 07:08:54,676 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:08:54,676 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Cluster state: [] 07:08:54,676 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:08:54,676 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:08:54,676 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:08:54,676 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:08:54,677 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:08:54,677 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): Cluster state: [] 07:08:54,677 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:08:54,677 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): RaftRPC message received RequestVoteReply{term=3, voteGranted=true}, my term is 3 07:08:54,677 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:08:54,677 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): handleRequestVoteReply: RequestVoteReply{term=3, voteGranted=true}, current voteCount: 1 07:08:54,677 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:08:54,677 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Election: Leader has following peers: [member-2-shard-cars-oper_testRemoveAllShardReplicas, member-3-shard-cars-oper_testRemoveAllShardReplicas] 07:08:54,677 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:08:54,677 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:54,678 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:54,678 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 07:08:54,678 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 07:08:54,678 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: became leader with frontend state for [] 07:08:54,678 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , received role change from Candidate to Leader 07:08:54,678 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testRemoveAllShardReplicas, leaderId=member-1-shard-cars-oper_testRemoveAllShardReplicas, leaderPayloadVersion=13] 07:08:54,678 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveAllShardReplicas from Candidate to Leader 07:08:54,678 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveAllShardReplicas is ready 07:08:54,679 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveAllShardReplicas status sync done false 07:08:54,679 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveAllShardReplicas status sync done false 07:08:54,680 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testRemoveAllShardReplicas, leaderId=member-1-shard-cars-oper_testRemoveAllShardReplicas, leaderPayloadVersion=13] 07:08:54,680 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveAllShardReplicas is ready 07:08:54,680 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testRemoveAllShardReplicas, leaderId=member-1-shard-cars-oper_testRemoveAllShardReplicas, leaderPayloadVersion=13] 07:08:54,680 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testRemoveAllShardReplicas is ready 07:08:54,680 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 07:08:54,680 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:54,680 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 07:08:54,680 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:54,703 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 07:08:54,704 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Starting new election term 3 07:08:54,704 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1} to peer member-3-shard-people-config_testRemoveAllShardReplicas 07:08:54,704 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1} to peer member-2-shard-people-config_testRemoveAllShardReplicas 07:08:54,705 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:08:54,705 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Cluster state: [] 07:08:54,705 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:08:54,705 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:08:54,705 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:08:54,705 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): Cluster state: [] 07:08:54,705 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:08:54,705 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:08:54,705 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:08:54,705 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:08:54,705 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:08:54,705 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:08:54,707 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): RaftRPC message received RequestVoteReply{term=3, voteGranted=true}, my term is 3 07:08:54,707 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): handleRequestVoteReply: RequestVoteReply{term=3, voteGranted=true}, current voteCount: 1 07:08:54,707 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Election: Leader has following peers: [member-3-shard-people-config_testRemoveAllShardReplicas, member-2-shard-people-config_testRemoveAllShardReplicas] 07:08:54,707 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:54,707 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:54,707 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 07:08:54,707 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 07:08:54,707 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: became leader with frontend state for [] 07:08:54,707 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testRemoveAllShardReplicas, leaderId=member-1-shard-people-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 07:08:54,707 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , received role change from Candidate to Leader 07:08:54,707 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveAllShardReplicas from Candidate to Leader 07:08:54,707 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 07:08:54,707 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} is now ready 07:08:54,708 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#720286852] 07:08:54,708 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:08:54,708 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#1590949546] 07:08:54,708 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#1590949546] 07:08:54,709 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveAllShardReplicas status sync done false 07:08:54,709 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveAllShardReplicas status sync done false 07:08:54,709 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testRemoveAllShardReplicas, leaderId=member-1-shard-people-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 07:08:54,709 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testRemoveAllShardReplicas, leaderId=member-1-shard-people-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 07:08:54,709 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 07:08:54,709 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 07:08:54,709 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 07:08:54,709 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:54,710 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1545984901] 07:08:54,710 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 07:08:54,710 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#1787224382] 07:08:54,710 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#1787224382] 07:08:54,710 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1410520283] 07:08:54,711 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 07:08:54,711 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:54,711 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 07:08:54,711 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#216494914] 07:08:54,711 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#216494914] 07:08:54,722 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@594e7b29] 07:08:54,723 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1 doCreateShard: shardId: member-1-shard-pets-config_testRemoveAllShardReplicas, memberNames: [MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], peerAddresses: {member-2-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas, member-3-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas}, isActiveMember: true 07:08:54,725 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - Shard created : member-1-shard-pets-config_testRemoveAllShardReplicas, persistent : true 07:08:54,726 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas-notifier#-656255590 created and ready for shard:member-1-shard-pets-config_testRemoveAllShardReplicas 07:08:54,726 AM [cluster-test-pekko.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas/shard-member-1:snapshot-read#934233119] 07:08:54,726 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - Starting recovery for member-1-shard-pets-config_testRemoveAllShardReplicas with journal batch size 1 07:08:54,727 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:54,727 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:54,727 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [pets] 07:08:54,727 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 07:08:54,727 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-pets-config_testRemoveAllShardReplicas is initialized 07:08:54,727 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:54,727 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , received role change from null to Follower 07:08:54,727 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 07:08:54,727 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testRemoveAllShardReplicas from null to Follower 07:08:54,731 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 07:08:54,735 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@3263023e] 07:08:54,736 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2 doCreateShard: shardId: member-2-shard-pets-config_testRemoveAllShardReplicas, memberNames: [MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], peerAddresses: {member-1-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas, member-3-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas}, isActiveMember: true 07:08:54,737 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Shard created : member-2-shard-pets-config_testRemoveAllShardReplicas, persistent : true 07:08:54,737 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Starting recovery for member-2-shard-pets-config_testRemoveAllShardReplicas with journal batch size 1 07:08:54,737 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas-notifier#-525143816 created and ready for shard:member-2-shard-pets-config_testRemoveAllShardReplicas 07:08:54,737 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas/shard-member-2:snapshot-read#2028241020] 07:08:54,739 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:54,739 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:54,739 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [pets] 07:08:54,739 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 07:08:54,739 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-pets-config_testRemoveAllShardReplicas is initialized 07:08:54,739 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:54,739 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testRemoveAllShardReplicas , received role change from null to Follower 07:08:54,739 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 07:08:54,739 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-pets-config_testRemoveAllShardReplicas from null to Follower 07:08:54,747 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@264c0247] 07:08:54,748 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3 doCreateShard: shardId: member-3-shard-pets-config_testRemoveAllShardReplicas, memberNames: [MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], peerAddresses: {member-2-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas, member-1-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas}, isActiveMember: true 07:08:54,748 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#720286852] 07:08:54,748 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : pets 07:08:54,748 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#-822568588] 07:08:54,749 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#-822568588] 07:08:54,749 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Shard created : member-3-shard-pets-config_testRemoveAllShardReplicas, persistent : true 07:08:54,749 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1545984901] 07:08:54,749 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : pets 07:08:54,749 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-1893312061] 07:08:54,749 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-1893312061] 07:08:54,749 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1410520283] 07:08:54,749 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : pets 07:08:54,749 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas/shard-member-3:snapshot-read#-188553967] 07:08:54,750 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas-notifier#2113944058 created and ready for shard:member-3-shard-pets-config_testRemoveAllShardReplicas 07:08:54,750 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Starting recovery for member-3-shard-pets-config_testRemoveAllShardReplicas with journal batch size 1 07:08:54,751 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:54,751 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:54,751 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 07:08:54,751 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:54,751 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-pets-config_testRemoveAllShardReplicas , received role change from null to Follower 07:08:54,750 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 300000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-pets-config_testRemoveAllShardReplicas, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 07:08:54,755 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [pets] 07:08:54,755 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-3-shard-pets-config_testRemoveAllShardReplicas is initialized 07:08:54,755 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-3-shard-pets-config_testRemoveAllShardReplicas is initialized - notifying 1 OnShardInitialized callbacks 07:08:54,755 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#1770583882] 07:08:54,755 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-pets-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 07:08:54,755 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#1770583882] 07:08:54,755 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-pets-config_testRemoveAllShardReplicas from null to Follower 07:08:54,757 AM [main] [INFO] ClusterAdminRpcService - Removing replicas for all shards 07:08:54,757 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people, pets] for data store config_testRemoveAllShardReplicas 07:08:54,757 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-3}] 07:08:54,757 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testRemoveAllShardReplicas 07:08:54,758 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=people, memberName=MemberName{name=member-3}] 07:08:54,758 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=pets, memberName=MemberName{name=member-3}] 07:08:54,758 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-3}] 07:08:54,758 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 07:08:54,758 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas, primaryVersion=13] 07:08:54,758 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 07:08:54,758 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for people: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas, primaryVersion=13] 07:08:54,758 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=pets, waitUntilReady=true] 07:08:54,758 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 18000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-pets-config_testRemoveAllShardReplicas, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=null, activeMember=true] 07:08:54,759 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas for shard member-3-shard-cars-config_testRemoveAllShardReplicas 07:08:54,759 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=people, memberName=MemberName{name=member-3}] 07:08:54,759 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 07:08:54,759 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas for shard member-3-shard-people-config_testRemoveAllShardReplicas 07:08:54,759 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas, primaryVersion=13] 07:08:54,760 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 07:08:54,760 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Found primary for people: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas, primaryVersion=13] 07:08:54,760 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas for shard member-3-shard-cars-oper_testRemoveAllShardReplicas 07:08:54,761 AM [cluster-test-shard-dispatcher-24] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onRemoveServer: RemoveServer{serverId='member-3-shard-cars-config_testRemoveAllShardReplicas'}, state: Idle 07:08:54,761 AM [cluster-test-shard-dispatcher-24] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]] 07:08:54,761 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]] 07:08:54,761 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]] 07:08:54,761 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onRemoveServer: RemoveServer{serverId='member-3-shard-people-config_testRemoveAllShardReplicas'}, state: Idle 07:08:54,761 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=a0178cc6-bba7-4b3d-b5b7-0403384bc343}, logIndex: 0, isSendImmediate: true 07:08:54,761 AM [cluster-test-shard-dispatcher-10] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onRemoveServer: RemoveServer{serverId='member-3-shard-cars-oper_testRemoveAllShardReplicas'}, state: Idle 07:08:54,762 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:54,762 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testRemoveAllShardReplicas: New server configuration : [ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]] 07:08:54,762 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas for shard member-3-shard-people-oper_testRemoveAllShardReplicas 07:08:54,762 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testRemoveAllShardReplicas 07:08:54,762 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]] 07:08:54,762 AM [cluster-test-shard-dispatcher-10] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]] 07:08:54,762 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testRemoveAllShardReplicas: AppendEntries{term=3, leaderId=member-1-shard-cars-config_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]]]} 07:08:54,762 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]] 07:08:54,762 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]] 07:08:54,762 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]] 07:08:54,762 AM [cluster-test-shard-dispatcher-24] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: Returning OK for operation RemoveServer{serverId='member-3-shard-cars-config_testRemoveAllShardReplicas'} 07:08:54,762 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=df97add1-89d7-4621-9622-449f08fb035d}, logIndex: 0, isSendImmediate: true 07:08:54,762 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=3900b9dc-fc00-4c07-906e-863d814fc8ea}, logIndex: 0, isSendImmediate: true 07:08:54,762 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:54,762 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-config_testRemoveAllShardReplicas 07:08:54,762 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:54,762 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Sending AppendEntries to follower member-2-shard-people-config_testRemoveAllShardReplicas: AppendEntries{term=3, leaderId=member-1-shard-people-config_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]]]} 07:08:54,762 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-oper_testRemoveAllShardReplicas 07:08:54,762 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testRemoveAllShardReplicas: Returning OK for operation RemoveServer{serverId='member-3-shard-people-config_testRemoveAllShardReplicas'} 07:08:54,762 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testRemoveAllShardReplicas: AppendEntries{term=3, leaderId=member-1-shard-cars-oper_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]]]} 07:08:54,762 AM [cluster-test-shard-dispatcher-10] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: Returning OK for operation RemoveServer{serverId='member-3-shard-cars-oper_testRemoveAllShardReplicas'} 07:08:54,763 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]] 07:08:54,763 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]] 07:08:54,763 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]] 07:08:54,764 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=member-1-shard-cars-config_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]]]} 07:08:54,764 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Number of entries to be appended = 1 07:08:54,764 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 07:08:54,764 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]] 07:08:54,764 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]] 07:08:54,764 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveAllShardReplicas} for shard cars 07:08:54,764 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard cars 07:08:54,764 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testRemoveAllShardReplicas, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas, votingState=VOTING]] 07:08:54,764 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-people-config_testRemoveAllShardReplicas} for shard people 07:08:54,764 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 07:08:54,764 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard people 07:08:54,764 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Log size is now 1 07:08:54,764 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveAllShardReplicas, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:54,764 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testRemoveAllShardReplicas} for shard cars 07:08:54,764 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 1 07:08:54,764 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully removed the replica shard cars 07:08:54,764 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 07:08:54,764 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 2 07:08:54,764 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 07:08:54,764 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 3 07:08:54,765 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]] 07:08:54,765 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=member-1-shard-people-config_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]]]} 07:08:54,765 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Number of entries to be appended = 1 07:08:54,765 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 07:08:54,765 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveAllShardReplicas status sync done true 07:08:54,765 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]] 07:08:54,765 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=member-1-shard-cars-oper_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]]]} 07:08:54,765 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]] 07:08:54,765 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Number of entries to be appended = 1 07:08:54,765 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 07:08:54,765 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]] 07:08:54,765 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]] 07:08:54,765 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onRemoveServer: RemoveServer{serverId='member-3-shard-people-oper_testRemoveAllShardReplicas'}, state: Idle 07:08:54,765 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-config_testRemoveAllShardReplicas, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas, votingState=VOTING]] 07:08:54,765 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Log size is now 1 07:08:54,765 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveAllShardReplicas, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:54,765 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testRemoveAllShardReplicas, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas, votingState=VOTING]] 07:08:54,765 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Log size is now 1 07:08:54,765 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: New server configuration : [ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]] 07:08:54,765 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:54,765 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]] 07:08:54,765 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]] 07:08:54,765 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testRemoveAllShardReplicas updated: matchIndex: 0, nextIndex: 1 07:08:54,765 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 07:08:54,765 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Setting last applied to 0 07:08:54,765 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 07:08:54,765 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]] 07:08:54,765 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=79b2cd31-2ccf-4419-9dde-777c4dac39c1}, logIndex: 0, isSendImmediate: true 07:08:54,765 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:54,766 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:54,766 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-oper_testRemoveAllShardReplicas 07:08:54,766 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]] has been successfully replicated to a majority of followers 07:08:54,766 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testRemoveAllShardReplicas: AppendEntries{term=3, leaderId=member-1-shard-people-oper_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]]]} 07:08:54,766 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 07:08:54,766 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: Returning OK for operation RemoveServer{serverId='member-3-shard-people-oper_testRemoveAllShardReplicas'} 07:08:54,766 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]] 07:08:54,766 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveAllShardReplicas status sync done true 07:08:54,766 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]] 07:08:54,766 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveAllShardReplicas status sync done true 07:08:54,767 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Commit index set to 0 07:08:54,767 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 07:08:54,767 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Setting last applied to 0 07:08:54,767 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 07:08:54,767 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]] 07:08:54,767 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=member-1-shard-people-oper_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]]]} 07:08:54,767 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 07:08:54,767 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Number of entries to be appended = 1 07:08:54,767 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 07:08:54,767 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]] 07:08:54,767 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]] 07:08:54,768 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-oper_testRemoveAllShardReplicas, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas, votingState=VOTING]] 07:08:54,768 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Log size is now 1 07:08:54,768 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveAllShardReplicas, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:54,768 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testRemoveAllShardReplicas updated: matchIndex: 0, nextIndex: 1 07:08:54,768 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]] 07:08:54,768 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 07:08:54,768 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Setting last applied to 0 07:08:54,768 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 07:08:54,768 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]] 07:08:54,768 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-people-oper_testRemoveAllShardReplicas} for shard people 07:08:54,768 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#216494914] with 27000 ms timeout 07:08:54,768 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:54,768 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully removed the replica shard people 07:08:54,768 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 07:08:54,768 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 4 07:08:54,768 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-config_testRemoveAllShardReplicas: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]] has been successfully replicated to a majority of followers 07:08:54,768 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: onShutDown 07:08:54,768 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 07:08:54,768 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]] 07:08:54,768 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard cars has been removed 07:08:54,768 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [pets, people] 07:08:54,768 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveAllShardReplicas status sync done true 07:08:54,769 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testRemoveAllShardReplicas 07:08:54,769 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 07:08:54,769 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Successfully deleted prior snapshots 07:08:54,769 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#216494914] 07:08:54,769 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Commit index set to 0 07:08:54,769 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 07:08:54,769 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Setting last applied to 0 07:08:54,769 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas#-727120548] with 27000 ms timeout 07:08:54,769 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 07:08:54,769 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]] 07:08:54,769 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard people has been removed 07:08:54,769 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 07:08:54,769 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [pets] 07:08:54,769 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: onShutDown 07:08:54,769 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testRemoveAllShardReplicas updated: matchIndex: 0, nextIndex: 1 07:08:54,769 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 07:08:54,769 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Setting last applied to 0 07:08:54,769 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 07:08:54,769 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 07:08:54,769 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]] 07:08:54,769 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:54,769 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: Successfully deleted prior snapshots 07:08:54,770 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]] has been successfully replicated to a majority of followers 07:08:54,770 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 07:08:54,770 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-3-shard-people-config_testRemoveAllShardReplicas 07:08:54,770 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas#-727120548] 07:08:54,770 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#-386378040] with 27000 ms timeout 07:08:54,770 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3 : Local Shard replica for shard people has been removed 07:08:54,770 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [cars] 07:08:54,770 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: onShutDown 07:08:54,770 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testRemoveAllShardReplicas 07:08:54,770 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Commit index set to 0 07:08:54,771 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 07:08:54,771 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Setting last applied to 0 07:08:54,771 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 07:08:54,771 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]] 07:08:54,771 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 07:08:54,771 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 07:08:54,771 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: Successfully deleted prior snapshots 07:08:54,771 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#-386378040] 07:08:54,771 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testRemoveAllShardReplicas updated: matchIndex: 0, nextIndex: 1 07:08:54,771 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 07:08:54,771 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Setting last applied to 0 07:08:54,771 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 07:08:54,771 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]] 07:08:54,771 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:54,771 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]] has been successfully replicated to a majority of followers 07:08:54,771 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 07:08:54,772 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#2043634457] with 27000 ms timeout 07:08:54,772 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Commit index set to 0 07:08:54,772 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 07:08:54,772 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Setting last applied to 0 07:08:54,772 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3 : Local Shard replica for shard cars has been removed 07:08:54,772 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 07:08:54,772 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [] 07:08:54,772 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]] 07:08:54,772 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 07:08:54,772 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 07:08:54,773 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: onShutDown 07:08:54,773 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testRemoveAllShardReplicas 07:08:54,773 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#2043634457] 07:08:54,774 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: Successfully deleted prior snapshots 07:08:55,54 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Up), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Up)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 07:08:55,54 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Follower): Received ElectionTimeout - switching to Candidate 07:08:55,54 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): Election: Candidate has following voting peers: [member-2-shard-pets-config_testRemoveAllShardReplicas, member-3-shard-pets-config_testRemoveAllShardReplicas] 07:08:55,54 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): Starting new election term 1 07:08:55,54 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): Sending RequestVote{term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1} to peer member-2-shard-pets-config_testRemoveAllShardReplicas 07:08:55,54 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): Sending RequestVote{term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1} to peer member-3-shard-pets-config_testRemoveAllShardReplicas 07:08:55,54 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:08:55,54 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 07:08:55,54 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:55,55 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , received role change from Follower to Candidate 07:08:55,55 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testRemoveAllShardReplicas from Follower to Candidate 07:08:55,56 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:08:55,56 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Cluster state: [] 07:08:55,56 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote{term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:08:55,56 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Term 1 in "RequestVote{term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:08:55,57 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote{term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 1, votedFor: null, lastIndex: -1, lastTerm: -1 07:08:55,57 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply{term=1, voteGranted=true} 07:08:55,57 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:08:55,57 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): Cluster state: [] 07:08:55,57 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote{term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:08:55,57 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): Term 1 in "RequestVote{term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:08:55,58 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote{term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 1, votedFor: null, lastIndex: -1, lastTerm: -1 07:08:55,58 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply{term=1, voteGranted=true} 07:08:55,59 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): RaftRPC message received RequestVoteReply{term=1, voteGranted=true}, my term is 1 07:08:55,59 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): handleRequestVoteReply: RequestVoteReply{term=1, voteGranted=true}, current voteCount: 1 07:08:55,59 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Election: Leader has following peers: [member-2-shard-pets-config_testRemoveAllShardReplicas, member-3-shard-pets-config_testRemoveAllShardReplicas] 07:08:55,59 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:55,59 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-pets-config_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:55,59 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 07:08:55,59 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 07:08:55,59 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: became leader with frontend state for [] 07:08:55,59 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , received role change from Candidate to Leader 07:08:55,59 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-pets-config_testRemoveAllShardReplicas, leaderId=member-1-shard-pets-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 07:08:55,60 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testRemoveAllShardReplicas from Candidate to Leader 07:08:55,60 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 07:08:55,61 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-pets-config_testRemoveAllShardReplicas, leaderId=member-1-shard-pets-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 07:08:55,61 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 07:08:55,61 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testRemoveAllShardReplicas status sync done false 07:08:55,61 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-pets-config_testRemoveAllShardReplicas status sync done false 07:08:55,62 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-pets-config_testRemoveAllShardReplicas, leaderId=member-1-shard-pets-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 07:08:55,62 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-3-shard-pets-config_testRemoveAllShardReplicas is ready - notifying 1 OnShardInitialized callbacks 07:08:55,62 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for pets: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas, primaryVersion=13] 07:08:55,62 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 07:08:55,62 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas for shard member-3-shard-pets-config_testRemoveAllShardReplicas 07:08:55,63 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-pets-config_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 07:08:55,63 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:55,63 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-pets-config_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 07:08:55,63 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-pets-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:08:55,63 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onRemoveServer: RemoveServer{serverId='member-3-shard-pets-config_testRemoveAllShardReplicas'}, state: Idle 07:08:55,64 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: New server configuration : [ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]] 07:08:55,64 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]] 07:08:55,64 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]] 07:08:55,64 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=7fdeaae0-3c2e-4270-a498-b84add69c6c0}, logIndex: 0, isSendImmediate: true 07:08:55,64 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:55,64 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): sendAppendEntries: 0 is present for follower member-2-shard-pets-config_testRemoveAllShardReplicas 07:08:55,64 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Sending AppendEntries to follower member-2-shard-pets-config_testRemoveAllShardReplicas: AppendEntries{term=1, leaderId=member-1-shard-pets-config_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]]]} 07:08:55,64 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: Returning OK for operation RemoveServer{serverId='member-3-shard-pets-config_testRemoveAllShardReplicas'} 07:08:55,64 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]] 07:08:55,65 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-pets-config_testRemoveAllShardReplicas} for shard pets 07:08:55,65 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard pets 07:08:55,65 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - onSuccess for shard pets, type Config 07:08:55,65 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 5 07:08:55,66 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#720286852] 07:08:55,66 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:08:55,67 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#1590949546] 07:08:55,67 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=member-1-shard-pets-config_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]]]} 07:08:55,67 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#1590949546] 07:08:55,67 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Number of entries to be appended = 1 07:08:55,67 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 07:08:55,67 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]] 07:08:55,67 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]] 07:08:55,67 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-pets-config_testRemoveAllShardReplicas, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas, votingState=VOTING]] 07:08:55,67 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Log size is now 1 07:08:55,67 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=member-2-shard-pets-config_testRemoveAllShardReplicas, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:55,68 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]] 07:08:55,68 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testRemoveAllShardReplicas status sync done true 07:08:55,68 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#720286852] 07:08:55,68 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 07:08:55,68 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-pets-config_testRemoveAllShardReplicas updated: matchIndex: 0, nextIndex: 1 07:08:55,68 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 07:08:55,68 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#-755534205] 07:08:55,68 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Setting last applied to 0 07:08:55,68 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#-755534205] 07:08:55,68 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 07:08:55,68 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]] 07:08:55,68 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:55,68 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]] has been successfully replicated to a majority of followers 07:08:55,68 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 07:08:55,68 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#720286852] 07:08:55,68 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : pets 07:08:55,68 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#-822568588] 07:08:55,69 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#-822568588] 07:08:55,69 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1545984901] 07:08:55,69 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 07:08:55,69 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#1787224382] 07:08:55,69 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#1787224382] 07:08:55,69 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1545984901] 07:08:55,69 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 07:08:55,69 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas#-1565320357] 07:08:55,69 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Commit index set to 0 07:08:55,69 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas#-1565320357] 07:08:55,69 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 07:08:55,69 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Setting last applied to 0 07:08:55,69 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 07:08:55,69 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]] 07:08:55,69 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 07:08:55,70 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1545984901] 07:08:55,70 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : pets 07:08:55,70 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-1893312061] 07:08:55,70 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#1770583882] with 27000 ms timeout 07:08:55,70 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard pets has been removed 07:08:55,70 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [] 07:08:55,70 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 07:08:55,70 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: Successfully deleted prior snapshots 07:08:55,70 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-1893312061] 07:08:55,70 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1410520283] 07:08:55,71 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 07:08:55,71 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard cars not found - shards present: [] 07:08:55,71 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: onShutDown 07:08:55,71 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1410520283] 07:08:55,71 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 07:08:55,71 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard people not found - shards present: [] 07:08:55,71 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1410520283] 07:08:55,71 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : pets 07:08:55,71 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard pets not found - shards present: [] 07:08:55,71 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-3-shard-pets-config_testRemoveAllShardReplicas 07:08:55,71 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 07:08:55,71 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#1770583882] 07:08:55,72 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 0 shards 07:08:55,72 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 07:08:55,72 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 07:08:55,72 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 07:08:55,72 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 0 shards 07:08:55,72 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 07:08:55,72 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 07:08:55,73 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:08:55,73 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:08:55,77 AM [cluster-test-pekko.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:08:55,77 AM [cluster-test-pekko.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:08:55,77 AM [cluster-test-pekko.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:08:55,78 AM [cluster-test-pekko.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:08:55,79 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 07:08:55,102 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 07:08:55,102 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-pets-config_testRemoveAllShardReplicas 07:08:55,102 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testRemoveAllShardReplicas 07:08:55,103 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveAllShardReplicas 07:08:55,103 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 3 shards 07:08:55,103 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: onShutDown 07:08:55,103 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: onShutDown 07:08:55,103 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: onShutDown 07:08:55,103 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testRemoveAllShardReplicas 07:08:55,103 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - Stopping Shard member-2-shard-people-config_testRemoveAllShardReplicas 07:08:55,104 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-2-shard-pets-config_testRemoveAllShardReplicas 07:08:55,105 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 07:08:55,105 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 07:08:55,105 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 07:08:55,105 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveAllShardReplicas 07:08:55,106 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveAllShardReplicas 07:08:55,106 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 07:08:55,106 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: onShutDown 07:08:55,106 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: onShutDown 07:08:55,106 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testRemoveAllShardReplicas 07:08:55,106 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testRemoveAllShardReplicas 07:08:55,107 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 07:08:55,107 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 07:08:55,108 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:08:55,108 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:08:55,544 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 07:08:56,94 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [3765145098590628681] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 07:08:56,124 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-4700198422083289251] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 07:08:56,126 AM [cluster-test-pekko.actor.default-dispatcher-19] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:08:56,126 AM [cluster-test-pekko.actor.default-dispatcher-19] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:08:56,128 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [01/31/2025 07:08:56.131] [cluster-test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:08:56,145 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 07:08:56,148 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-pets-config_testRemoveAllShardReplicas 07:08:56,148 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testRemoveAllShardReplicas 07:08:56,148 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testRemoveAllShardReplicas 07:08:56,148 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 3 shards 07:08:56,148 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: onShutDown 07:08:56,148 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Initiating leader transfer 07:08:56,148 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: onShutDown 07:08:56,148 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Initiating leader transfer 07:08:56,149 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3f3e25a 07:08:56,149 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@51237fdd 07:08:56,149 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-pets-config_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3f3e25a 07:08:56,149 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-config_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@51237fdd 07:08:56,149 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 07:08:56,149 AM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 07:08:56,149 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Attempting to transfer leadership 07:08:56,149 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Attempting to transfer leadership 07:08:56,149 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-pets-config_testRemoveAllShardReplicas, leaderId=null, leaderPayloadVersion=13] 07:08:56,149 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testRemoveAllShardReplicas: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:56,149 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveAllShardReplicas: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:56,149 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testRemoveAllShardReplicas, leaderId=null, leaderPayloadVersion=13] 07:08:56,149 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: onShutDown 07:08:56,149 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Initiating leader transfer 07:08:56,149 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testRemoveAllShardReplicas, leaderId=null, leaderPayloadVersion=13] 07:08:56,149 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6271004d 07:08:56,149 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-config_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6271004d 07:08:56,149 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 07:08:56,149 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Attempting to transfer leadership 07:08:56,149 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveAllShardReplicas: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:56,544 AM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveAllShardReplicas: leader transfer aborted 07:08:56,544 AM [cluster-test-shard-dispatcher-23] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveAllShardReplicas: Failed to transfer leadership in 396.0 ms 07:08:56,545 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: In unpauseLeader 07:08:56,545 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: leader transfer failed - sending PoisonPill 07:08:56,545 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Leadership transfer expired 07:08:56,546 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testRemoveAllShardReplicas 07:08:56,624 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveAllShardReplicas: leader transfer aborted 07:08:56,624 AM [cluster-test-shard-dispatcher-22] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveAllShardReplicas: Failed to transfer leadership in 474.7 ms 07:08:56,624 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: In unpauseLeader 07:08:56,624 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: leader transfer failed - sending PoisonPill 07:08:56,624 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Leadership transfer expired 07:08:56,624 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-1-shard-people-config_testRemoveAllShardReplicas 07:08:56,654 AM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testRemoveAllShardReplicas: leader transfer aborted 07:08:56,654 AM [cluster-test-shard-dispatcher-23] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testRemoveAllShardReplicas: Failed to transfer leadership in 506.1 ms 07:08:56,655 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: In unpauseLeader 07:08:56,655 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: leader transfer failed - sending PoisonPill 07:08:56,655 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Leadership transfer expired 07:08:56,655 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Stopping Shard member-1-shard-pets-config_testRemoveAllShardReplicas 07:08:56,656 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 07:08:56,656 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 07:08:56,656 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 07:08:56,657 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testRemoveAllShardReplicas 07:08:56,657 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testRemoveAllShardReplicas 07:08:56,657 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 07:08:56,657 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: onShutDown 07:08:56,657 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Initiating leader transfer 07:08:56,657 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5a71b23c 07:08:56,657 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardDataTree - member-1-shard-people-oper_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5a71b23c 07:08:56,657 AM [cluster-test-shard-dispatcher-10] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 07:08:56,657 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testRemoveAllShardReplicas, leaderId=null, leaderPayloadVersion=13] 07:08:56,657 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Attempting to transfer leadership 07:08:56,657 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveAllShardReplicas: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:56,657 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: onShutDown 07:08:56,657 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Initiating leader transfer 07:08:56,658 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1421fb9d 07:08:56,658 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testRemoveAllShardReplicas, leaderId=null, leaderPayloadVersion=13] 07:08:56,658 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1421fb9d 07:08:56,658 AM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 07:08:56,658 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Attempting to transfer leadership 07:08:56,658 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveAllShardReplicas: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:57,194 AM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveAllShardReplicas: leader transfer aborted 07:08:57,194 AM [cluster-test-shard-dispatcher-23] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveAllShardReplicas: Failed to transfer leadership in 536.8 ms 07:08:57,194 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: In unpauseLeader 07:08:57,194 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: leader transfer failed - sending PoisonPill 07:08:57,194 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Leadership transfer expired 07:08:57,195 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testRemoveAllShardReplicas 07:08:57,246 AM [cluster-test-shard-dispatcher-10] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveAllShardReplicas: leader transfer aborted 07:08:57,246 AM [cluster-test-shard-dispatcher-10] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveAllShardReplicas: Failed to transfer leadership in 588.2 ms 07:08:57,246 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: In unpauseLeader 07:08:57,246 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: leader transfer failed - sending PoisonPill 07:08:57,246 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Leadership transfer expired 07:08:57,247 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testRemoveAllShardReplicas 07:08:57,247 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 07:08:57,247 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 07:08:57,248 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:08:57,248 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:08:57,250 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [01/31/2025 07:08:57.252] [cluster-test-pekko.actor.internal-dispatcher-7] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:08:57,271 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:08:57,279 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-2404337937612154935] 07:08:57,279 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 07:08:57,280 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 07:08:57,280 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 07:08:57,282 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:08:57,282 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:08:57,291 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:57,293 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:57,293 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 07:08:57,293 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 07:08:57,293 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 07:08:57,295 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplicaFailures 07:08:57,301 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 07:08:57,302 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#-1125131416] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:57,304 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 07:08:57,304 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testAddShardReplicaFailures 07:08:57,304 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:57,304 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures 07:08:57,307 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testAddShardReplicaFailures: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplicaFailures, generation=0} 07:08:57,308 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@3b5d4693} 07:08:57,308 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplicaFailures, generation=0} started 07:08:57,309 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:57,309 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testAddShardReplicaFailures 07:08:57,309 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:57,309 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-cars-config_testAddShardReplicaFailures, persistent : true 07:08:57,310 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures#-398402116] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:57,310 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testAddShardReplicaFailures with journal batch size 1 07:08:57,310 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures-notifier#1392191659 created and ready for shard:member-1-shard-cars-config_testAddShardReplicaFailures 07:08:57,310 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures-notifier#1392191659] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:57,310 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures 07:08:57,310 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures/shard-member-1:snapshot-read#-798741847] 07:08:57,310 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures to pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures 07:08:57,310 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testAddShardReplicaFailures on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures on shard member-1-shard-cars-config_testAddShardReplicaFailures whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures#-398402116] 07:08:57,311 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures/shard-member-1:snapshot-read#-798741847] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:57,311 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:57,312 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:57,312 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 07:08:57,312 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:57,312 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Follower): Received TimeoutNow - switching to Candidate 07:08:57,312 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:57,312 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Candidate): Election: Candidate has following voting peers: [] 07:08:57,312 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Candidate): Starting new election term 1 07:08:57,312 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:08:57,312 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 07:08:57,312 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:57,312 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 07:08:57,312 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testAddShardReplicaFailures is initialized 07:08:57,312 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , received role change from null to Follower 07:08:57,312 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , received role change from Follower to Candidate 07:08:57,313 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Candidate): Received ElectionTimeout 07:08:57,313 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Leader): Election: Leader has following peers: [] 07:08:57,313 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 07:08:57,313 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplicaFailures: onLeadershipChange, isLeader: true, hasLeader : true 07:08:57,313 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: became leader with frontend state for [] 07:08:57,313 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , received role change from Candidate to Leader 07:08:57,314 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplicaFailures 07:08:57,316 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 07:08:57,317 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures 07:08:57,317 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testAddShardReplicaFailures, leaderId=member-1-shard-cars-config_testAddShardReplicaFailures, leaderPayloadVersion=13] 07:08:57,317 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures#-2078794142] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:57,317 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddShardReplicaFailures from Candidate to Leader 07:08:57,317 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplicaFailures is ready 07:08:57,318 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 07:08:57,319 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testAddShardReplicaFailures 07:08:57,319 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:57,319 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures 07:08:57,320 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testAddShardReplicaFailures: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplicaFailures, generation=0} 07:08:57,321 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplicaFailures, generation=0} started 07:08:57,322 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@27ec4a1f} 07:08:57,323 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard people 07:08:57,323 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: onAddShardReplica: AddShardReplica[ShardName=people] 07:08:57,323 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: No module configuration exists for shard people 07:08:57,324 AM [cluster-test-pekko.actor.default-dispatcher-18] [ERROR] ClusterAdminRpcService - Failed to add replica for shard people java.lang.IllegalArgumentException: No module configuration exists for shard people at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onAddShardReplica(ShardManager.java:1193) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:245) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 07:08:57,324 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplicaFailures, generation=0} 07:08:57,324 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testAddShardReplicaFailures 07:08:57,324 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 07:08:57,324 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: onShutDown 07:08:57,324 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@3da6e5b6 07:08:57,324 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplicaFailures: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@3da6e5b6 07:08:57,324 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:57,324 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testAddShardReplicaFailures 07:08:57,325 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testAddShardReplicaFailures 07:08:57,325 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures 07:08:57,325 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures to pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures 07:08:57,325 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testAddShardReplicaFailures on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures on shard member-1-shard-cars-oper_testAddShardReplicaFailures whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures#-1824736421] 07:08:57,325 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 07:08:57,325 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 07:08:57,325 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-cars-oper_testAddShardReplicaFailures, persistent : true 07:08:57,326 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures#-1824736421] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:57,326 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures/shard-member-1:snapshot-read#1805963121] 07:08:57,326 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testAddShardReplicaFailures with journal batch size 1 07:08:57,326 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures-notifier#-1240097859 created and ready for shard:member-1-shard-cars-oper_testAddShardReplicaFailures 07:08:57,326 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures-notifier#-1240097859] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:57,326 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures/shard-member-1:snapshot-read#1805963121] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:57,326 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplicaFailures, generation=0} 07:08:57,326 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#-1125131416] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#-1125131416] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:57,327 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testAddShardReplicaFailures 07:08:57,327 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#-1125131416] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#-1125131416] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:57,327 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 1 shards 07:08:57,327 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:57,327 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:57,327 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 07:08:57,327 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testAddShardReplicaFailures is initialized 07:08:57,327 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 07:08:57,327 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:57,327 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , received role change from null to Follower 07:08:57,327 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: onShutDown 07:08:57,327 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures (Follower): Received TimeoutNow - switching to Candidate 07:08:57,327 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures (Candidate): Election: Candidate has following voting peers: [] 07:08:57,327 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures (Candidate): Starting new election term 1 07:08:57,327 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:08:57,327 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 07:08:57,327 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:57,327 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , received role change from Follower to Candidate 07:08:57,328 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testAddShardReplicaFailures 07:08:57,328 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 07:08:57,328 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 07:08:57,329 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:08:57,329 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:08:57,330 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 07:08:57,351 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:08:57,361 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-1953211842983976093] 07:08:57,362 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 07:08:57,364 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 07:08:57,364 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 07:08:57,373 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:57,375 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:57,377 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 07:08:57,380 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 07:08:57,384 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1449952078] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:57,384 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:08:57,384 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:08:57,386 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 07:08:57,386 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 07:08:57,387 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 07:08:57,387 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 07:08:57,387 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testAddShardReplica 07:08:57,388 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:57,389 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 07:08:57,391 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplica, generation=0} 07:08:57,392 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@467e4e25} 07:08:57,392 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplica, generation=0} started 07:08:57,392 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:57,392 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testAddShardReplica 07:08:57,393 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 07:08:57,393 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 07:08:57,393 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testAddShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica on shard member-1-shard-cars-config_testAddShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-2097920714] 07:08:57,393 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-config_testAddShardReplica, persistent : true 07:08:57,393 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-2097920714] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:57,393 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-notifier#787735612 created and ready for shard:member-1-shard-cars-config_testAddShardReplica 07:08:57,393 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-notifier#787735612] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:57,393 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testAddShardReplica with journal batch size 1 07:08:57,393 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/shard-member-1:snapshot-read#1273320274] 07:08:57,394 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/shard-member-1:snapshot-read#1273320274] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:57,394 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:57,394 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:57,394 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 07:08:57,394 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testAddShardReplica is initialized 07:08:57,399 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:08:57,399 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:57,399 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , received role change from null to Follower 07:08:57,399 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Follower): Received TimeoutNow - switching to Candidate 07:08:57,399 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Candidate): Election: Candidate has following voting peers: [] 07:08:57,399 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica (Candidate): Starting new election term 1 07:08:57,399 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:08:57,399 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:08:57,399 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:57,399 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , received role change from Follower to Candidate 07:08:57,400 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Candidate): Received ElectionTimeout 07:08:57,400 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Election: Leader has following peers: [] 07:08:57,400 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 07:08:57,400 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 07:08:57,400 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: became leader with frontend state for [] 07:08:57,400 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , received role change from Candidate to Leader 07:08:57,400 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplica 07:08:57,400 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testAddShardReplica, leaderId=member-1-shard-cars-config_testAddShardReplica, leaderPayloadVersion=13] 07:08:57,400 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddShardReplica from Candidate to Leader 07:08:57,400 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 07:08:57,442 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1449952078] 07:08:57,442 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:08:57,442 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-2097920714] 07:08:57,443 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:57,444 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:57,446 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplica 07:08:57,447 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 07:08:57,448 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#1595715164] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:57,448 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 07:08:57,448 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testAddShardReplica 07:08:57,449 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:57,449 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 07:08:57,452 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplica, generation=0} 07:08:57,452 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@32b55823} 07:08:57,452 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplica, generation=0} started 07:08:57,453 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:57,453 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testAddShardReplica 07:08:57,454 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 07:08:57,454 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 07:08:57,454 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testAddShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica on shard member-1-shard-cars-oper_testAddShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-654813042] 07:08:57,454 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-oper_testAddShardReplica, persistent : true 07:08:57,454 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-654813042] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:57,454 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/shard-member-1:snapshot-read#398949005] 07:08:57,454 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-notifier#1971798606 created and ready for shard:member-1-shard-cars-oper_testAddShardReplica 07:08:57,454 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testAddShardReplica with journal batch size 1 07:08:57,454 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-notifier#1971798606] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:57,454 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/shard-member-1:snapshot-read#398949005] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:57,455 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:57,455 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:57,455 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 07:08:57,455 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testAddShardReplica is initialized 07:08:57,455 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:08:57,455 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:57,455 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from null to Follower 07:08:57,455 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Follower): Received TimeoutNow - switching to Candidate 07:08:57,455 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Candidate): Election: Candidate has following voting peers: [] 07:08:57,455 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica (Candidate): Starting new election term 1 07:08:57,455 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:08:57,455 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:08:57,455 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:57,455 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from Follower to Candidate 07:08:57,456 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Candidate): Received ElectionTimeout 07:08:57,456 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Election: Leader has following peers: [] 07:08:57,456 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 07:08:57,456 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 07:08:57,456 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: became leader with frontend state for [] 07:08:57,456 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from Candidate to Leader 07:08:57,456 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testAddShardReplica 07:08:57,456 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testAddShardReplica, leaderId=member-1-shard-cars-oper_testAddShardReplica, leaderPayloadVersion=13] 07:08:57,456 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddShardReplica from Candidate to Leader 07:08:57,456 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 07:08:57,503 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#1595715164] 07:08:57,503 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 07:08:57,503 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-654813042] 07:08:57,513 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:08:57,523 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [2913752259598775779] 07:08:57,523 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 07:08:57,525 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:08:57,525 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 07:08:57,526 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:08:57,526 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:08:57,562 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:57,565 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:57,569 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 07:08:57,578 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1322263234] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:57,579 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 07:08:57,583 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 07:08:57,609 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} 07:08:57,612 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@394d870} 07:08:57,612 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} started 07:08:57,613 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:57,613 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:57,616 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:57,620 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplica 07:08:57,623 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#302123235] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:57,624 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 07:08:57,626 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 07:08:57,628 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} 07:08:57,628 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@d533672} 07:08:57,639 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} started 07:08:57,639 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:57,650 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 07:08:57,654 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:57,654 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 07:08:57,654 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:57,654 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 07:08:57,654 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 07:08:58,406 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:58,406 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 07:08:58,406 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 07:08:58,406 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 07:08:58,406 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 07:08:58,406 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:58,406 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 07:08:58,406 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 07:08:58,406 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 07:08:58,442 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 07:08:58,442 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 07:08:58,443 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 07:08:58,443 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica] 07:08:58,446 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 07:08:58,446 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-2097920714, primaryVersion=13] 07:08:58,449 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-2097920714 for shard member-2-shard-cars-config_testAddShardReplica 07:08:58,450 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-2-shard-cars-config_testAddShardReplica, persistent : true 07:08:58,450 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-1901339912] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:58,451 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-notifier#836574547 created and ready for shard:member-2-shard-cars-config_testAddShardReplica 07:08:58,451 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/shard-member-2:snapshot-read#1428627410] 07:08:58,451 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-notifier#836574547] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:58,451 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/shard-member-2:snapshot-read#1428627410] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:58,451 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testAddShardReplica with journal batch size 1 07:08:58,452 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: onAddServer: AddServer [newServerId=member-2-shard-cars-config_testAddShardReplica, newServerAddress=null, votingMember=true], state: Idle 07:08:58,452 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: Initiating AddServer [newServerId=member-2-shard-cars-config_testAddShardReplica, newServerAddress=null, votingMember=true] 07:08:58,453 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 07:08:58,453 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-config_testAddShardReplica 07:08:58,453 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: lastSequenceNumber prior to capture: 1 07:08:58,453 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 07:08:58,453 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: Initiating capture snapshot for new server member-2-shard-cars-config_testAddShardReplica 07:08:58,453 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:58,453 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: CaptureSnapshotReply received by actor 07:08:58,453 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:58,453 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@7226eeb3, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testAddShardReplica, ServerConfigPayload=null] 07:08:58,453 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 07:08:58,453 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendInstallSnapshot 07:08:58,453 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:08:58,454 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:58,454 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): next snapshot chunk size for follower member-2-shard-cars-config_testAddShardReplica: 229 07:08:58,454 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 07:08:58,454 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testAddShardReplica is initialized 07:08:58,454 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testAddShardReplica, member-2-shard-cars-config_testAddShardReplica), Chunk: 1/1 07:08:58,454 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 07:08:58,454 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Snapshot success - sequence number: -1 07:08:58,454 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: SnapshotComplete received 07:08:58,454 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddShardReplica , received role change from null to Follower 07:08:58,454 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: handled a snapshot response 07:08:58,454 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: handled a journal response 07:08:58,454 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplica 07:08:58,454 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testAddShardReplica from null to Follower 07:08:58,455 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 07:08:58,455 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleInstallSnapshot: InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]} 07:08:58,455 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 07:08:58,455 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Snapshot received from leader: member-1-shard-cars-config_testAddShardReplica 07:08:58,455 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testAddShardReplica, leaderId=member-1-shard-cars-config_testAddShardReplica, leaderPayloadVersion=-1] 07:08:58,456 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 07:08:58,456 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done false 07:08:58,456 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: Applying snapshot on follower: org.opendaylight.controller.cluster.io.FileBackedOutputStream$1@33d0cd69 07:08:58,457 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Converted InstallSnapshot from leader: member-1-shard-cars-config_testAddShardReplica to state 07:08:58,457 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 07:08:58,457 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 07:08:58,457 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Snapshot success - sequence number: 1 07:08:58,457 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Moving last applied index from -1 to -1 07:08:58,457 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: Applying snapshot 07:08:58,457 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 07:08:58,457 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-config_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-DTCL-publisher#917404999] with name member-2-shard-cars-config_testAddShardReplica-DTCL-publisher 07:08:58,457 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-DTCL-publisher#917404999] 07:08:58,458 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 07:08:58,457 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-2-shard-cars-config_testAddShardReplica: state snapshot applied in 391.8 μs 07:08:58,458 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-DTCL-publisher#917404999] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:58,458 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: Done applying snapshot 07:08:58,458 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleInstallSnapshot returning: InstallSnapshotReply{term=1, followerId=member-2-shard-cars-config_testAddShardReplica, chunkIndex=1, success=true} 07:08:58,458 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: SnapshotComplete received 07:08:58,458 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-config_testAddShardReplica-DTCL-publisher: 129.2 μs 07:08:58,458 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: handled a snapshot response 07:08:58,458 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: handled a journal response 07:08:58,459 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleInstallSnapshotReply: InstallSnapshotReply{term=1, followerId=member-2-shard-cars-config_testAddShardReplica, chunkIndex=1, success=true} 07:08:58,459 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Snapshot successfully installed on follower member-2-shard-cars-config_testAddShardReplica (last chunk 1) - matchIndex set to -1, nextIndex set to 0 07:08:58,459 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 07:08:58,459 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-cars-config_testAddShardReplica] 07:08:58,459 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]] 07:08:58,459 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 07:08:58,459 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 07:08:58,459 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=672e04da-690e-4e96-bce3-81c57090cb91}, logIndex: 0, isSendImmediate: true 07:08:58,459 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:58,459 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testAddShardReplica 07:08:58,459 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testAddShardReplica: AppendEntries{term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]]]} 07:08:58,459 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: Returning OK for operation AddServer [newServerId=member-2-shard-cars-config_testAddShardReplica, newServerAddress=null, votingMember=true] 07:08:58,460 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 07:08:58,461 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]]]} 07:08:58,461 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 07:08:58,461 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testAddShardReplica} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-2097920714 07:08:58,461 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 07:08:58,461 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard cars 07:08:58,461 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 07:08:58,461 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 07:08:58,461 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testAddShardReplica, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, votingState=VOTING]] 07:08:58,461 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Log size is now 1 07:08:58,461 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=member-2-shard-cars-config_testAddShardReplica, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:58,462 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-config_testAddShardReplica 07:08:58,462 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [cars] 07:08:58,462 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 07:08:58,462 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 07:08:58,462 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 07:08:58,462 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1322263234] 07:08:58,462 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Re-initializing to Follower with previous leaderId member-1-shard-cars-config_testAddShardReplica 07:08:58,462 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testAddShardReplica updated: matchIndex: 0, nextIndex: 1 07:08:58,462 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 07:08:58,462 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Setting last applied to 0 07:08:58,462 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Moving last applied index from -1 to 0 07:08:58,462 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 07:08:58,462 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testAddShardReplica, leaderId=member-1-shard-cars-config_testAddShardReplica, leaderPayloadVersion=13] 07:08:58,462 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 07:08:58,462 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:58,462 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 07:08:58,462 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done true 07:08:58,462 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:58,462 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 07:08:58,462 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 07:08:58,462 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-1901339912] 07:08:58,462 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] has been successfully replicated to a majority of followers 07:08:58,462 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-1901339912] 07:08:58,462 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=0 07:08:58,462 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Successfully deleted prior snapshots 07:08:58,463 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#302123235] 07:08:58,463 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 07:08:58,463 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Local shard cars not found - shards present: [] 07:08:58,463 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 07:08:58,463 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 07:08:58,463 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Commit index set to 0 07:08:58,463 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 07:08:58,463 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 0 07:08:58,463 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Moving last applied index from -1 to 0 07:08:58,463 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 07:08:58,463 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=0 07:08:58,463 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 07:08:58,463 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica] 07:08:58,463 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done false 07:08:58,464 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 07:08:58,464 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-654813042, primaryVersion=13] 07:08:58,466 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-654813042 for shard member-2-shard-cars-oper_testAddShardReplica 07:08:58,467 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#917371528] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:58,467 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - Shard created : member-2-shard-cars-oper_testAddShardReplica, persistent : true 07:08:58,467 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: onAddServer: AddServer [newServerId=member-2-shard-cars-oper_testAddShardReplica, newServerAddress=null, votingMember=true], state: Idle 07:08:58,467 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: Initiating AddServer [newServerId=member-2-shard-cars-oper_testAddShardReplica, newServerAddress=null, votingMember=true] 07:08:58,467 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 07:08:58,467 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-notifier#-1048938085 created and ready for shard:member-2-shard-cars-oper_testAddShardReplica 07:08:58,467 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-oper_testAddShardReplica 07:08:58,467 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: lastSequenceNumber prior to capture: 1 07:08:58,468 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-notifier#-1048938085] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:58,468 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 07:08:58,468 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: Initiating capture snapshot for new server member-2-shard-cars-oper_testAddShardReplica 07:08:58,468 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/shard-member-2:snapshot-read#1863219304] 07:08:58,468 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/shard-member-2:snapshot-read#1863219304] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:58,468 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testAddShardReplica with journal batch size 1 07:08:58,468 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: CaptureSnapshotReply received by actor 07:08:58,468 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@671d3823, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testAddShardReplica, ServerConfigPayload=null] 07:08:58,468 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 07:08:58,468 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendInstallSnapshot 07:08:58,468 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): next snapshot chunk size for follower member-2-shard-cars-oper_testAddShardReplica: 229 07:08:58,469 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-oper_testAddShardReplica, member-2-shard-cars-oper_testAddShardReplica), Chunk: 1/1 07:08:58,469 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 07:08:58,469 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Snapshot success - sequence number: -1 07:08:58,469 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: SnapshotComplete received 07:08:58,469 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: handled a snapshot response 07:08:58,469 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: handled a journal response 07:08:58,469 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:08:58,470 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:08:58,470 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 07:08:58,470 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testAddShardReplica is initialized 07:08:58,470 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:08:58,470 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:58,470 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddShardReplica , received role change from null to Follower 07:08:58,470 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 07:08:58,470 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleInstallSnapshot: InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]} 07:08:58,470 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 07:08:58,470 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Snapshot received from leader: member-1-shard-cars-oper_testAddShardReplica 07:08:58,474 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testAddShardReplica 07:08:58,474 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: Applying snapshot on follower: org.opendaylight.controller.cluster.io.FileBackedOutputStream$1@7aa59b97 07:08:58,474 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done false 07:08:58,474 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testAddShardReplica, leaderId=member-1-shard-cars-oper_testAddShardReplica, leaderPayloadVersion=-1] 07:08:58,474 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testAddShardReplica from null to Follower 07:08:58,474 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 07:08:58,474 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Converted InstallSnapshot from leader: member-1-shard-cars-oper_testAddShardReplica to state 07:08:58,474 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 07:08:58,475 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 07:08:58,475 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Snapshot success - sequence number: 1 07:08:58,475 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Moving last applied index from -1 to -1 07:08:58,475 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: Applying snapshot 07:08:58,475 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 07:08:58,475 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher#-1513741673] 07:08:58,475 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-oper_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher#-1513741673] with name member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher 07:08:58,475 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 07:08:58,475 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher#-1513741673] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:58,475 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher: 71.89 μs 07:08:58,476 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testAddShardReplica: state snapshot applied in 842.4 μs 07:08:58,476 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: Done applying snapshot 07:08:58,476 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleInstallSnapshot returning: InstallSnapshotReply{term=1, followerId=member-2-shard-cars-oper_testAddShardReplica, chunkIndex=1, success=true} 07:08:58,476 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: SnapshotComplete received 07:08:58,476 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: handled a snapshot response 07:08:58,476 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: handled a journal response 07:08:58,476 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleInstallSnapshotReply: InstallSnapshotReply{term=1, followerId=member-2-shard-cars-oper_testAddShardReplica, chunkIndex=1, success=true} 07:08:58,476 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Snapshot successfully installed on follower member-2-shard-cars-oper_testAddShardReplica (last chunk 1) - matchIndex set to -1, nextIndex set to 0 07:08:58,476 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 07:08:58,476 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-cars-oper_testAddShardReplica] 07:08:58,476 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]] 07:08:58,477 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 07:08:58,477 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 07:08:58,477 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=4e19a0cb-e1ae-4d42-a31a-99378187cbcd}, logIndex: 0, isSendImmediate: true 07:08:58,477 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:58,477 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-oper_testAddShardReplica 07:08:58,477 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testAddShardReplica: AppendEntries{term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]]]} 07:08:58,477 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: Returning OK for operation AddServer [newServerId=member-2-shard-cars-oper_testAddShardReplica, newServerAddress=null, votingMember=true] 07:08:58,477 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 07:08:58,478 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]]]} 07:08:58,478 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 07:08:58,478 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 07:08:58,478 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 07:08:58,478 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 07:08:58,478 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testAddShardReplica, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, votingState=VOTING]] 07:08:58,478 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Log size is now 1 07:08:58,478 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=member-2-shard-cars-oper_testAddShardReplica, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:08:58,478 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testAddShardReplica} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-654813042 07:08:58,479 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 07:08:58,479 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard cars 07:08:58,479 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testAddShardReplica updated: matchIndex: 0, nextIndex: 1 07:08:58,479 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 07:08:58,479 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Setting last applied to 0 07:08:58,479 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Moving last applied index from -1 to 0 07:08:58,479 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 07:08:58,479 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:08:58,479 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] has been successfully replicated to a majority of followers 07:08:58,479 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-oper_testAddShardReplica 07:08:58,479 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=0 07:08:58,479 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars] 07:08:58,479 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testAddShardReplica, leaderId=member-1-shard-cars-oper_testAddShardReplica, leaderPayloadVersion=13] 07:08:58,480 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 07:08:58,480 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 07:08:58,480 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#302123235] 07:08:58,479 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 07:08:58,480 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Re-initializing to Follower with previous leaderId member-1-shard-cars-oper_testAddShardReplica 07:08:58,480 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done true 07:08:58,480 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 07:08:58,480 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 07:08:58,480 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:08:58,480 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 07:08:58,480 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#917371528] 07:08:58,480 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Successfully deleted prior snapshots 07:08:58,480 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#917371528] 07:08:58,480 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 0 07:08:58,480 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 07:08:58,480 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 0 07:08:58,480 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Moving last applied index from -1 to 0 07:08:58,480 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 07:08:58,480 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=0 07:08:58,480 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done false 07:08:58,489 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:08:58,499 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-7086739839258536791] 07:08:58,499 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 07:08:58,501 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:08:58,501 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 07:08:58,505 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:08:58,505 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:08:58,512 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:58,516 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:58,518 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 07:08:58,522 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 07:08:58,527 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1586897756] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:58,528 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 07:08:58,532 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} 07:08:58,533 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@481fc429} 07:08:58,533 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} started 07:08:58,534 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:58,534 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:58,535 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:08:58,538 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplica 07:08:58,539 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-855226891] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:08:58,539 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 07:08:58,542 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 07:08:58,542 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} 07:08:58,546 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} started 07:08:58,546 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@d1d534e} 07:08:58,546 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:08:58,547 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:58,547 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 07:08:58,547 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 07:08:58,547 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 07:08:58,547 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:58,547 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 07:08:58,547 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 07:08:58,547 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 07:08:58,555 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 07:08:58,557 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 07:08:58,563 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:58,563 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 07:08:58,563 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:58,563 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 07:08:58,563 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:08:58,563 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 07:08:58,563 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:08:58,563 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 07:08:58,785 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done true 07:08:59,86 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done true 07:09:00,445 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 07:09:00,445 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:00,445 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 07:09:00,445 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 07:09:00,445 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 07:09:00,446 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:00,447 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 07:09:00,447 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 07:09:00,447 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 07:09:00,449 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:00,449 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 07:09:00,450 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:00,450 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 07:09:00,534 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 07:09:00,538 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:00,538 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 07:09:00,538 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 07:09:00,538 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 07:09:00,538 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:00,538 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 07:09:00,538 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 07:09:00,538 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 07:09:00,554 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 07:09:00,554 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 07:09:00,555 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 07:09:00,555 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica, visitedAddresses: [pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica] 07:09:00,557 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 07:09:00,557 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-2097920714, primaryVersion=13] 07:09:00,559 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-2097920714 for shard member-3-shard-cars-config_testAddShardReplica 07:09:00,560 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#-499672942] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:00,560 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - Shard created : member-3-shard-cars-config_testAddShardReplica, persistent : true 07:09:00,561 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#-1598946858 created and ready for shard:member-3-shard-cars-config_testAddShardReplica 07:09:00,561 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/shard-member-3:snapshot-read#-1239258265] 07:09:00,561 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#-1598946858] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:00,562 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/shard-member-3:snapshot-read#-1239258265] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:00,562 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testAddShardReplica with journal batch size 1 07:09:00,562 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: onAddServer: AddServer [newServerId=member-3-shard-cars-config_testAddShardReplica, newServerAddress=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica, votingMember=true], state: Idle 07:09:00,562 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: Initiating AddServer [newServerId=member-3-shard-cars-config_testAddShardReplica, newServerAddress=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica, votingMember=true] 07:09:00,562 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-config_testAddShardReplica 07:09:00,562 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: lastSequenceNumber prior to capture: 3 07:09:00,563 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 07:09:00,563 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: Initiating capture snapshot for new server member-3-shard-cars-config_testAddShardReplica 07:09:00,563 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: CaptureSnapshotReply received by actor 07:09:00,563 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@4de7aea5, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testAddShardReplica, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 07:09:00,563 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 07:09:00,564 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendInstallSnapshot 07:09:00,564 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): next snapshot chunk size for follower member-3-shard-cars-config_testAddShardReplica: 229 07:09:00,564 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testAddShardReplica, member-3-shard-cars-config_testAddShardReplica), Chunk: 1/1 07:09:00,564 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 07:09:00,564 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Snapshot success - sequence number: -1 07:09:00,564 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: SnapshotComplete received 07:09:00,564 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:00,565 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:00,565 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: handled a snapshot response 07:09:00,565 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 07:09:00,565 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized 07:09:00,565 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: handled a journal response 07:09:00,565 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:09:00,565 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:00,565 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , received role change from null to Follower 07:09:00,565 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplica 07:09:00,565 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testAddShardReplica from null to Follower 07:09:00,566 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]}" message is greater than follower's term 0 - updating term 07:09:00,566 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleInstallSnapshot: InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]} 07:09:00,566 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 07:09:00,566 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Snapshot received from leader: member-1-shard-cars-config_testAddShardReplica 07:09:00,566 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testAddShardReplica, leaderId=member-1-shard-cars-config_testAddShardReplica, leaderPayloadVersion=-1] 07:09:00,566 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 07:09:00,567 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done false 07:09:00,567 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Applying snapshot on follower: org.opendaylight.controller.cluster.io.FileBackedOutputStream$1@72ffb8d3 07:09:00,567 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Converted InstallSnapshot from leader: member-1-shard-cars-config_testAddShardReplica to state 07:09:00,567 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 07:09:00,567 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 07:09:00,567 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Snapshot success - sequence number: 1 07:09:00,567 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from -1 to 0 07:09:00,567 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testAddShardReplica, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, votingState=VOTING]] 07:09:00,567 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Applying snapshot 07:09:00,567 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 07:09:00,568 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-config_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#1210562335] with name member-3-shard-cars-config_testAddShardReplica-DTCL-publisher 07:09:00,568 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-3-shard-cars-config_testAddShardReplica: state snapshot applied in 326.4 μs 07:09:00,568 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#1210562335] 07:09:00,568 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Done applying snapshot 07:09:00,568 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleInstallSnapshot returning: InstallSnapshotReply{term=1, followerId=member-3-shard-cars-config_testAddShardReplica, chunkIndex=1, success=true} 07:09:00,568 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#1210562335] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:00,568 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: SnapshotComplete received 07:09:00,568 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: handled a journal response 07:09:00,568 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 07:09:00,568 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: handled a snapshot response 07:09:00,568 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-config_testAddShardReplica-DTCL-publisher: 132.1 μs 07:09:00,569 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleInstallSnapshotReply: InstallSnapshotReply{term=1, followerId=member-3-shard-cars-config_testAddShardReplica, chunkIndex=1, success=true} 07:09:00,569 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Snapshot successfully installed on follower member-3-shard-cars-config_testAddShardReplica (last chunk 1) - matchIndex set to 0, nextIndex set to 1 07:09:00,569 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 07:09:00,569 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-3-shard-cars-config_testAddShardReplica] 07:09:00,569 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]] 07:09:00,569 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 07:09:00,569 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 07:09:00,569 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=78393871-bf85-41e0-930e-4991180313f9}, logIndex: 1, isSendImmediate: true 07:09:00,569 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:00,569 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testAddShardReplica 07:09:00,569 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testAddShardReplica: AppendEntries{term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]]]} 07:09:00,569 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:00,569 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testAddShardReplica 07:09:00,570 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testAddShardReplica: AppendEntries{term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]]]} 07:09:00,570 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: Returning OK for operation AddServer [newServerId=member-3-shard-cars-config_testAddShardReplica, newServerAddress=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica, votingMember=true] 07:09:00,570 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 07:09:00,571 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]]]} 07:09:00,571 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testAddShardReplica} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-2097920714 07:09:00,571 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 07:09:00,571 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully added the replica shard cars 07:09:00,571 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 07:09:00,571 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 07:09:00,571 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 07:09:00,571 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testAddShardReplica, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, votingState=VOTING]] 07:09:00,571 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Log size is now 2 07:09:00,571 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=member-2-shard-cars-config_testAddShardReplica, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:00,572 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-config_testAddShardReplica 07:09:00,572 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [cars] 07:09:00,572 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 07:09:00,572 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 07:09:00,572 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1586897756] 07:09:00,572 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 07:09:00,572 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 07:09:00,572 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: Successfully deleted prior snapshots 07:09:00,572 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#-499672942] 07:09:00,571 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]]]} 07:09:00,572 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 07:09:00,572 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#-499672942] 07:09:00,572 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 07:09:00,572 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 07:09:00,572 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 07:09:00,572 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testAddShardReplica, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, votingState=VOTING]] 07:09:00,572 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Log size is now 1 07:09:00,572 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=member-3-shard-cars-config_testAddShardReplica, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:00,573 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testAddShardReplica updated: matchIndex: 1, nextIndex: 2 07:09:00,573 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 07:09:00,573 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Setting last applied to 1 07:09:00,573 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Moving last applied index from 0 to 1 07:09:00,573 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 07:09:00,573 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testAddShardReplica, leaderId=member-1-shard-cars-config_testAddShardReplica, leaderPayloadVersion=13] 07:09:00,573 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 07:09:00,573 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 0 for term 1 07:09:00,573 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:09:00,573 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] has been successfully replicated to a majority of followers 07:09:00,573 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=1 07:09:00,574 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Commit index set to 1 07:09:00,574 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 07:09:00,574 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 1 07:09:00,574 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Moving last applied index from 0 to 1 07:09:00,574 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 07:09:00,574 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 0 for term 1 07:09:00,574 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=1 07:09:00,574 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 07:09:00,574 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testAddShardReplica updated: matchIndex: 1, nextIndex: 2 07:09:00,574 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:09:00,574 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-855226891] 07:09:00,574 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 07:09:00,574 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: Local shard cars not found - shards present: [] 07:09:00,574 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 07:09:00,574 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 07:09:00,574 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Re-initializing to Follower with previous leaderId member-1-shard-cars-config_testAddShardReplica 07:09:00,574 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 07:09:00,575 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 07:09:00,575 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica, visitedAddresses: [pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica] 07:09:00,575 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 07:09:00,575 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:00,575 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done true 07:09:00,575 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Commit index set to 1 07:09:00,575 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 07:09:00,575 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 1 07:09:00,575 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from 0 to 1 07:09:00,575 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 07:09:00,576 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=1 07:09:00,576 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 07:09:00,576 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done false 07:09:00,576 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-654813042, primaryVersion=13] 07:09:00,578 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-3-shard-cars-oper_testAddShardReplica, persistent : true 07:09:00,578 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#1729347719] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:00,578 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/shard-member-3:snapshot-read#-1599053448] 07:09:00,578 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-notifier#-1175701461 created and ready for shard:member-3-shard-cars-oper_testAddShardReplica 07:09:00,578 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/shard-member-3:snapshot-read#-1599053448] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:00,578 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testAddShardReplica with journal batch size 1 07:09:00,579 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-notifier#-1175701461] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:00,579 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-654813042 for shard member-3-shard-cars-oper_testAddShardReplica 07:09:00,581 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:00,581 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:00,581 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:09:00,581 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:00,581 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testAddShardReplica , received role change from null to Follower 07:09:00,581 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 07:09:00,581 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testAddShardReplica is initialized 07:09:00,581 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testAddShardReplica 07:09:00,581 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testAddShardReplica from null to Follower 07:09:00,582 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: onAddServer: AddServer [newServerId=member-3-shard-cars-oper_testAddShardReplica, newServerAddress=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica, votingMember=true], state: Idle 07:09:00,582 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: Initiating AddServer [newServerId=member-3-shard-cars-oper_testAddShardReplica, newServerAddress=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica, votingMember=true] 07:09:00,582 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-oper_testAddShardReplica 07:09:00,582 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: lastSequenceNumber prior to capture: 3 07:09:00,582 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 07:09:00,582 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: Initiating capture snapshot for new server member-3-shard-cars-oper_testAddShardReplica 07:09:00,583 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: CaptureSnapshotReply received by actor 07:09:00,583 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@471fc5d5, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testAddShardReplica, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 07:09:00,583 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 07:09:00,583 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendInstallSnapshot 07:09:00,583 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): next snapshot chunk size for follower member-3-shard-cars-oper_testAddShardReplica: 229 07:09:00,583 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-oper_testAddShardReplica, member-3-shard-cars-oper_testAddShardReplica), Chunk: 1/1 07:09:00,584 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 07:09:00,584 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Snapshot success - sequence number: -1 07:09:00,584 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: SnapshotComplete received 07:09:00,584 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]}" message is greater than follower's term 0 - updating term 07:09:00,584 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: handled a snapshot response 07:09:00,585 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: handled a journal response 07:09:00,585 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleInstallSnapshot: InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]} 07:09:00,585 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 07:09:00,585 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Snapshot received from leader: member-1-shard-cars-oper_testAddShardReplica 07:09:00,585 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testAddShardReplica, leaderId=member-1-shard-cars-oper_testAddShardReplica, leaderPayloadVersion=-1] 07:09:00,585 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 07:09:00,585 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: Applying snapshot on follower: org.opendaylight.controller.cluster.io.FileBackedOutputStream$1@6d32ac01 07:09:00,585 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done false 07:09:00,586 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Converted InstallSnapshot from leader: member-1-shard-cars-oper_testAddShardReplica to state 07:09:00,586 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 07:09:00,587 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 07:09:00,587 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Snapshot success - sequence number: 1 07:09:00,587 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Moving last applied index from -1 to 0 07:09:00,587 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-oper_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testAddShardReplica, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, votingState=VOTING]] 07:09:00,587 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: Applying snapshot 07:09:00,587 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-3-shard-cars-oper_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 07:09:00,587 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-oper_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher#-1505903766] with name member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher 07:09:00,587 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher#-1505903766] 07:09:00,587 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testAddShardReplica: state snapshot applied in 311.9 μs 07:09:00,587 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: Done applying snapshot 07:09:00,587 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleInstallSnapshot returning: InstallSnapshotReply{term=1, followerId=member-3-shard-cars-oper_testAddShardReplica, chunkIndex=1, success=true} 07:09:00,587 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 07:09:00,587 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher: 120.3 μs 07:09:00,588 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher#-1505903766] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:00,588 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: SnapshotComplete received 07:09:00,588 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: handled a snapshot response 07:09:00,588 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: handled a journal response 07:09:00,588 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleInstallSnapshotReply: InstallSnapshotReply{term=1, followerId=member-3-shard-cars-oper_testAddShardReplica, chunkIndex=1, success=true} 07:09:00,588 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Snapshot successfully installed on follower member-3-shard-cars-oper_testAddShardReplica (last chunk 1) - matchIndex set to 0, nextIndex set to 1 07:09:00,588 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 07:09:00,588 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-3-shard-cars-oper_testAddShardReplica] 07:09:00,588 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]] 07:09:00,588 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 07:09:00,588 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 07:09:00,589 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=2ede58b6-b9fb-40d4-930e-4a417c0be798}, logIndex: 1, isSendImmediate: true 07:09:00,589 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:00,589 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testAddShardReplica 07:09:00,589 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testAddShardReplica: AppendEntries{term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]]]} 07:09:00,589 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:00,589 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testAddShardReplica 07:09:00,589 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries{term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]]]} 07:09:00,589 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: Returning OK for operation AddServer [newServerId=member-3-shard-cars-oper_testAddShardReplica, newServerAddress=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica, votingMember=true] 07:09:00,590 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 07:09:00,590 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]]]} 07:09:00,590 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testAddShardReplica} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-654813042 07:09:00,590 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 07:09:00,590 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully added the replica shard cars 07:09:00,590 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 07:09:00,590 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 07:09:00,590 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 07:09:00,590 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-oper_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testAddShardReplica, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, votingState=VOTING]] 07:09:00,590 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Log size is now 2 07:09:00,590 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=member-2-shard-cars-oper_testAddShardReplica, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:00,590 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 07:09:00,591 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-oper_testAddShardReplica 07:09:00,591 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [cars] 07:09:00,591 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 07:09:00,591 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testAddShardReplica updated: matchIndex: 1, nextIndex: 2 07:09:00,591 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 07:09:00,591 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 07:09:00,591 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-855226891] 07:09:00,591 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Setting last applied to 1 07:09:00,591 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Moving last applied index from 0 to 1 07:09:00,591 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 07:09:00,591 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]]]} 07:09:00,591 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 0 for term 1 07:09:00,591 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 07:09:00,591 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 07:09:00,591 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 07:09:00,591 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 07:09:00,591 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:09:00,591 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] has been successfully replicated to a majority of followers 07:09:00,591 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-oper_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testAddShardReplica, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, votingState=VOTING]] 07:09:00,591 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Log size is now 1 07:09:00,591 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=1 07:09:00,591 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=member-3-shard-cars-oper_testAddShardReplica, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:00,591 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 07:09:00,591 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Successfully deleted prior snapshots 07:09:00,591 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#1729347719] 07:09:00,592 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testAddShardReplica, leaderId=member-1-shard-cars-oper_testAddShardReplica, leaderPayloadVersion=13] 07:09:00,592 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#1729347719] 07:09:00,592 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 07:09:00,592 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 1 07:09:00,592 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 07:09:00,592 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 1 07:09:00,592 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Moving last applied index from 0 to 1 07:09:00,592 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 07:09:00,592 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 0 for term 1 07:09:00,592 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=1 07:09:00,592 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testAddShardReplica updated: matchIndex: 1, nextIndex: 2 07:09:00,592 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 07:09:00,593 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 07:09:00,593 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 07:09:00,593 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1322263234] 07:09:00,593 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Re-initializing to Follower with previous leaderId member-1-shard-cars-oper_testAddShardReplica 07:09:00,593 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 07:09:00,593 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 07:09:00,593 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-1901339912] 07:09:00,593 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:00,593 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-1901339912] 07:09:00,593 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 1 07:09:00,593 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 07:09:00,593 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 1 07:09:00,593 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Moving last applied index from 0 to 1 07:09:00,593 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 07:09:00,593 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=1 07:09:00,593 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#302123235] 07:09:00,593 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 07:09:00,594 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#917371528] 07:09:00,594 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#917371528] 07:09:00,594 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done true 07:09:00,594 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done false 07:09:00,634 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1322263234] 07:09:00,635 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 07:09:00,635 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 07:09:00,635 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 07:09:00,646 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} is not yet known 07:09:00,657 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, nanosAgo=68832, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} 07:09:00,664 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-2097920714], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 07:09:00,667 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-2097920714], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 07:09:00,690 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-2097920714], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 24.55 ms 07:09:00,692 AM [cluster-test-shard-dispatcher-20] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: allocating new open transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 07:09:00,711 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: readying transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 07:09:00,712 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 ready 07:09:00,716 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: transitioned member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 to ready 07:09:00,717 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 initiating direct canCommit 07:09:00,717 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Validating transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 07:09:00,717 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 validated 07:09:00,717 AM [cluster-test-shard-dispatcher-20] [DEBUG] CommitCohort - Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 07:09:00,718 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 initiating direct preCommit 07:09:00,718 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Preparing transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 07:09:00,721 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 candidate ready 07:09:00,726 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: canCommit - candidate rootPath: / 07:09:00,735 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: canCommit - messages: [] 07:09:00,735 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 07:09:00,735 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 prepared 07:09:00,735 AM [cluster-test-shard-dispatcher-20] [DEBUG] CommitCohort - Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 07:09:00,735 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 initiating direct commit 07:09:00,736 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Starting commit for transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 07:09:00,741 AM [cluster-test-shard-dispatcher-20] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 235 07:09:00,741 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 07:09:00,742 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 07:09:00,742 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Replicate message: identifier: member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, logIndex: 2, isSendImmediate: true 07:09:00,742 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 07:09:00,742 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-config_testAddShardReplica 07:09:00,743 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testAddShardReplica: AppendEntries{term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}]]} 07:09:00,743 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 07:09:00,743 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-config_testAddShardReplica 07:09:00,743 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testAddShardReplica: AppendEntries{term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}]]} 07:09:00,743 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 submitted to persistence 07:09:00,746 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}]]} 07:09:00,746 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 07:09:00,746 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 07:09:00,746 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 07:09:00,746 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Append entry to log Simple{size=235} 07:09:00,746 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 07:09:00,747 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Log size is now 2 07:09:00,747 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=member-2-shard-cars-config_testAddShardReplica, logLastIndex=2, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:00,747 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 07:09:00,748 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testAddShardReplica updated: matchIndex: 2, nextIndex: 3 07:09:00,748 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 07:09:00,748 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Setting last applied to 2 07:09:00,748 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Moving last applied index from 1 to 2 07:09:00,748 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Applying state for log index 2 data Simple{size=235} 07:09:00,748 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}]]} 07:09:00,748 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 07:09:00,748 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 07:09:00,748 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Append entry to log Simple{size=235} 07:09:00,748 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 07:09:00,748 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Log size is now 2 07:09:00,748 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=member-3-shard-cars-config_testAddShardReplica, logLastIndex=2, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:00,748 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 07:09:00,749 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done true 07:09:00,749 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Resuming commit of transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 07:09:00,751 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendMetadata - member-1-shard-cars-config_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@205ffbce}, purged=MutableUnsignedLongSet{size=0}} 07:09:00,751 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 07:09:00,752 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 07:09:00,753 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: commit - successfulFromPrevious: [] 07:09:00,753 AM [cluster-test-shard-dispatcher-20] [DEBUG] CommitCohort - Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 07:09:00,756 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-config_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-DTCL-publisher#-304682959] with name member-1-shard-cars-config_testAddShardReplica-DTCL-publisher 07:09:00,756 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-DTCL-publisher#-304682959] 07:09:00,757 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 07:09:00,757 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-1-shard-cars-config_testAddShardReplica-DTCL-publisher: 121.9 μs 07:09:00,757 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-DTCL-publisher#-304682959] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:00,768 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1586897756] 07:09:00,768 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 07:09:00,768 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 07:09:00,768 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 07:09:00,770 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 1 for term 1 07:09:00,770 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:09:00,770 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testAddShardReplica updated: matchIndex: 2, nextIndex: 3 07:09:00,770 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:09:00,770 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=2 07:09:00,770 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Commit index set to 2 07:09:00,770 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 07:09:00,770 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 2 07:09:00,770 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Moving last applied index from 1 to 2 07:09:00,770 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Applying state for log index 2 data Simple{size=235} 07:09:00,771 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Commit index set to 2 07:09:00,771 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 07:09:00,771 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 2 07:09:00,771 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from 1 to 2 07:09:00,771 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Applying state for log index 2 data Simple{size=235} 07:09:00,771 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-config_testAddShardReplica: Applying foreign transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 07:09:00,771 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-3-shard-cars-config_testAddShardReplica: Applying foreign transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 07:09:00,772 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: purging transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 07:09:00,773 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@1b2f4f9a}, purged=MutableUnsignedLongSet{size=0}} 07:09:00,773 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@1daf41ba}, purged=MutableUnsignedLongSet{size=0}} 07:09:00,773 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 07:09:00,773 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 07:09:00,773 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 07:09:00,773 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 07:09:00,773 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 1 for term 1 07:09:00,773 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 1 for term 1 07:09:00,773 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=2 07:09:00,773 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=2 07:09:00,773 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 07:09:00,773 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-config_testAddShardReplica-DTCL-publisher: 84.97 μs 07:09:00,774 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 07:09:00,774 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-config_testAddShardReplica-DTCL-publisher: 40.89 μs 07:09:00,775 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 07:09:00,775 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 07:09:00,775 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Replicate message: identifier: member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, logIndex: 3, isSendImmediate: false 07:09:00,775 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-config_testAddShardReplica 07:09:00,775 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testAddShardReplica: AppendEntries{term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}]]} 07:09:00,775 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 3 is present for follower member-3-shard-cars-config_testAddShardReplica 07:09:00,775 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testAddShardReplica: AppendEntries{term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}]]} 07:09:00,775 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} is not yet known 07:09:00,775 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, nanosAgo=7940, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} 07:09:00,777 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 07:09:00,778 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}]]} 07:09:00,778 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 07:09:00,778 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 07:09:00,778 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56} 07:09:00,778 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 07:09:00,778 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Log size is now 2 07:09:00,778 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=member-2-shard-cars-config_testAddShardReplica, logLastIndex=3, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:00,778 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 07:09:00,778 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}]]} 07:09:00,778 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 07:09:00,778 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 07:09:00,778 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56} 07:09:00,779 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 07:09:00,779 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-2097920714], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 07:09:00,779 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-2097920714], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 07:09:00,779 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-2097920714], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 497.2 μs 07:09:00,779 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Log size is now 2 07:09:00,779 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=member-3-shard-cars-config_testAddShardReplica, logLastIndex=3, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:00,780 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 07:09:00,780 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testAddShardReplica updated: matchIndex: 3, nextIndex: 4 07:09:00,780 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 07:09:00,780 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Setting last applied to 3 07:09:00,780 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Moving last applied index from 2 to 3 07:09:00,780 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56} 07:09:00,780 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: replication of member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda/0x00007f971c93f580@67cdec4b 07:09:00,781 AM [cluster-test-shard-dispatcher-20] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: finished purging transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 07:09:00,781 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 07:09:00,781 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 07:09:00,781 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 07:09:00,781 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:09:00,781 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=3 07:09:00,783 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testAddShardReplica updated: matchIndex: 3, nextIndex: 4 07:09:00,783 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:09:00,783 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Commit index set to 3 07:09:00,783 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 07:09:00,783 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 3 07:09:00,783 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Moving last applied index from 2 to 3 07:09:00,783 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56} 07:09:00,783 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 07:09:00,783 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 07:09:00,783 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 07:09:00,783 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=3 07:09:00,784 AM [cluster-test-shard-dispatcher-20] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: allocating new open snapshot member-3-datastore-config_testAddShardReplica-fe-0-txn-0-1 07:09:00,784 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Commit index set to 3 07:09:00,784 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 07:09:00,784 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 3 07:09:00,784 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from 2 to 3 07:09:00,784 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56} 07:09:00,784 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 07:09:00,784 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 07:09:00,784 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 07:09:00,784 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=3 07:09:00,791 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-855226891] 07:09:00,791 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 07:09:00,791 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 07:09:00,791 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 07:09:00,793 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} is not yet known 07:09:00,793 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, nanosAgo=13380, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} 07:09:00,794 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-654813042], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 07:09:00,794 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-654813042], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 07:09:00,795 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-654813042], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 656.9 μs 07:09:00,797 AM [cluster-test-shard-dispatcher-20] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: allocating new open transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 07:09:00,797 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: readying transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 07:09:00,797 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 ready 07:09:00,797 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: transitioned member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 to ready 07:09:00,797 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 initiating direct canCommit 07:09:00,797 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Validating transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 07:09:00,797 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 validated 07:09:00,797 AM [cluster-test-shard-dispatcher-20] [DEBUG] CommitCohort - Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 07:09:00,797 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 initiating direct preCommit 07:09:00,797 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Preparing transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 07:09:00,797 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 candidate ready 07:09:00,797 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: canCommit - candidate rootPath: / 07:09:00,797 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: canCommit - messages: [] 07:09:00,797 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 07:09:00,797 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 prepared 07:09:00,797 AM [cluster-test-shard-dispatcher-20] [DEBUG] CommitCohort - Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 07:09:00,797 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 initiating direct commit 07:09:00,797 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Starting commit for transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 07:09:00,798 AM [cluster-test-shard-dispatcher-20] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 233 07:09:00,798 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 07:09:00,798 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 07:09:00,798 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Replicate message: identifier: member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, logIndex: 2, isSendImmediate: true 07:09:00,798 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 07:09:00,798 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-oper_testAddShardReplica 07:09:00,798 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testAddShardReplica: AppendEntries{term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}]]} 07:09:00,798 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 07:09:00,798 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-oper_testAddShardReplica 07:09:00,798 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries{term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}]]} 07:09:00,798 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 submitted to persistence 07:09:00,799 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 07:09:00,799 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}]]} 07:09:00,799 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 07:09:00,799 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 07:09:00,799 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Append entry to log Simple{size=233} 07:09:00,799 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 07:09:00,800 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Log size is now 2 07:09:00,800 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=member-2-shard-cars-oper_testAddShardReplica, logLastIndex=2, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:00,800 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}]]} 07:09:00,800 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 07:09:00,800 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 07:09:00,800 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Append entry to log Simple{size=233} 07:09:00,800 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 07:09:00,800 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 07:09:00,800 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Log size is now 2 07:09:00,800 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=member-3-shard-cars-oper_testAddShardReplica, logLastIndex=2, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:00,801 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 07:09:00,801 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testAddShardReplica updated: matchIndex: 2, nextIndex: 3 07:09:00,801 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done true 07:09:00,801 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 07:09:00,801 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Setting last applied to 2 07:09:00,801 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Moving last applied index from 1 to 2 07:09:00,801 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Applying state for log index 2 data Simple{size=233} 07:09:00,801 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Resuming commit of transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 07:09:00,801 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendMetadata - member-1-shard-cars-oper_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@4d7b24f6}, purged=MutableUnsignedLongSet{size=0}} 07:09:00,801 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 07:09:00,801 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: Committed transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 07:09:00,801 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: commit - successfulFromPrevious: [] 07:09:00,801 AM [cluster-test-shard-dispatcher-20] [DEBUG] CommitCohort - Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 07:09:00,802 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-oper_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher#280482622] with name member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher 07:09:00,802 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher#280482622] 07:09:00,802 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 07:09:00,802 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 1 for term 1 07:09:00,802 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:09:00,802 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher: 170.3 μs 07:09:00,802 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testAddShardReplica updated: matchIndex: 2, nextIndex: 3 07:09:00,802 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 07:09:00,802 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=2 07:09:00,803 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 2 07:09:00,803 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 07:09:00,803 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 2 07:09:00,803 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Moving last applied index from 1 to 2 07:09:00,803 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Applying state for log index 2 data Simple{size=233} 07:09:00,803 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testAddShardReplica: Applying foreign transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 07:09:00,804 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@1f7669}, purged=MutableUnsignedLongSet{size=0}} 07:09:00,804 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 07:09:00,804 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: Committed transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 07:09:00,804 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 1 for term 1 07:09:00,804 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=2 07:09:00,804 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher#280482622] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:00,805 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 07:09:00,805 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher: 113.4 μs 07:09:00,805 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#302123235] 07:09:00,805 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 07:09:00,805 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 07:09:00,806 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: purging transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 07:09:00,806 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 07:09:00,805 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 2 07:09:00,806 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 07:09:00,806 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 07:09:00,806 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 2 07:09:00,806 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Moving last applied index from 1 to 2 07:09:00,806 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Applying state for log index 2 data Simple{size=233} 07:09:00,806 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Replicate message: identifier: member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, logIndex: 3, isSendImmediate: false 07:09:00,806 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testAddShardReplica: Applying foreign transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 07:09:00,807 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-3-shard-cars-oper_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@5adacc67}, purged=MutableUnsignedLongSet{size=0}} 07:09:00,807 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 07:09:00,807 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: Committed transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 07:09:00,807 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 1 for term 1 07:09:00,807 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=2 07:09:00,807 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 07:09:00,808 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 07:09:00,808 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher: 62.91 μs 07:09:00,808 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 3 is present for follower member-3-shard-cars-oper_testAddShardReplica 07:09:00,808 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries{term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]} 07:09:00,808 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 07:09:00,809 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} is not yet known 07:09:00,809 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, nanosAgo=9370, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} 07:09:00,809 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]} 07:09:00,809 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 07:09:00,809 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 07:09:00,809 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Append entry to log PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54} 07:09:00,809 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 07:09:00,809 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Log size is now 2 07:09:00,809 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=member-3-shard-cars-oper_testAddShardReplica, logLastIndex=3, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:00,809 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 07:09:00,810 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-654813042], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 07:09:00,810 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-654813042], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 07:09:00,810 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testAddShardReplica updated: matchIndex: 3, nextIndex: 4 07:09:00,810 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 07:09:00,810 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Setting last applied to 3 07:09:00,810 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Moving last applied index from 2 to 3 07:09:00,810 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54} 07:09:00,810 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: replication of member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda/0x00007f971c93f580@881adc2 07:09:00,810 AM [cluster-test-shard-dispatcher-20] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: finished purging transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 07:09:00,810 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 07:09:00,810 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: Purged transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 07:09:00,810 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 07:09:00,810 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-654813042], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 311.5 μs 07:09:00,810 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:09:00,810 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=3 07:09:00,811 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 3 07:09:00,811 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 07:09:00,811 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 3 07:09:00,811 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Moving last applied index from 2 to 3 07:09:00,811 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54} 07:09:00,811 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 07:09:00,811 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: Purged transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 07:09:00,811 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 07:09:00,811 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=3 07:09:00,811 AM [cluster-test-shard-dispatcher-20] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: allocating new open snapshot member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 07:09:00,813 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1449952078] 07:09:00,813 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:09:00,814 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-2097920714] 07:09:00,814 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-2097920714] 07:09:00,814 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#1595715164] 07:09:00,814 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 07:09:00,814 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-654813042] 07:09:00,814 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-654813042] 07:09:00,814 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1322263234] 07:09:00,814 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 07:09:00,814 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-1901339912] 07:09:00,815 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-1901339912] 07:09:00,815 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#302123235] 07:09:00,815 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 07:09:00,815 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#917371528] 07:09:00,815 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#917371528] 07:09:00,865 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#917371528] 07:09:00,916 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#917371528] 07:09:00,967 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#917371528] 07:09:01,17 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#917371528] 07:09:01,68 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#917371528] 07:09:01,119 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#917371528] 07:09:01,169 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#917371528] 07:09:01,220 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#917371528] 07:09:01,271 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#917371528] 07:09:01,322 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#917371528] 07:09:01,343 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-oper_testAddShardReplica 07:09:01,343 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testAddShardReplica: AppendEntries{term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]} 07:09:01,345 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]} 07:09:01,345 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 07:09:01,345 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 07:09:01,345 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Append entry to log PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54} 07:09:01,345 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 07:09:01,345 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Log size is now 2 07:09:01,345 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 3 07:09:01,345 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=member-2-shard-cars-oper_testAddShardReplica, logLastIndex=3, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:01,345 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 07:09:01,345 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 3 07:09:01,346 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Moving last applied index from 2 to 3 07:09:01,346 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54} 07:09:01,346 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 07:09:01,346 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: Purged transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 07:09:01,346 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 07:09:01,346 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 07:09:01,347 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=3 07:09:01,347 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testAddShardReplica updated: matchIndex: 3, nextIndex: 4 07:09:01,347 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:09:01,372 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#917371528] 07:09:01,373 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1586897756] 07:09:01,373 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 07:09:01,373 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#-499672942] 07:09:01,373 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#-499672942] 07:09:01,374 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-855226891] 07:09:01,374 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 07:09:01,374 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#1729347719] 07:09:01,374 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#1729347719] 07:09:01,374 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} 07:09:01,375 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testAddShardReplica 07:09:01,375 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node [pekko://cluster-test@127.0.0.1:2557] as [Down] 07:09:01,375 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 07:09:01,375 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onShutDown 07:09:01,377 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testAddShardReplica 07:09:01,377 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 07:09:01,377 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 07:09:01,377 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} 07:09:01,379 AM [cluster-test-pekko.actor.default-dispatcher-11] [WARN] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Received gossip where this member has been downed, from [pekko://cluster-test@127.0.0.1:2558] 07:09:01,379 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Node has been marked as DOWN. Shutting down myself 07:09:01,379 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testAddShardReplica 07:09:01,379 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 1 shards 07:09:01,380 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: onShutDown 07:09:01,381 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Shutting down... 07:09:01,381 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Successfully shut down 07:09:01,381 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testAddShardReplica 07:09:01,381 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 07:09:01,382 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberRemoved: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:01,382 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 07:09:01,383 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:09:01,384 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:09:01,385 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] CoordinatedShutdown - Running CoordinatedShutdown with reason [ClusterDowningReason] 07:09:01,389 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:01,390 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:01,390 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:01,390 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:01,391 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 07:09:01,414 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:09:01,427 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-7333825138389519978] 07:09:01,427 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 07:09:01,431 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Registered cluster JMX MBean [pekko:type=Cluster] 07:09:01,431 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 07:09:01,431 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:09:01,431 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:09:01,441 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:01,443 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:01,445 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 07:09:01,455 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 07:09:01,455 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1915835000] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:01,456 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onSnapshotOffer: ShardManagerSnapshot [ShardList = [cars] ] 07:09:01,456 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: adding shard cars 07:09:01,457 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 07:09:01,457 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testAddShardReplica 07:09:01,459 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@5e169812} 07:09:01,460 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} started 07:09:01,460 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1915835000] 07:09:01,460 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:01,460 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testAddShardReplica 07:09:01,461 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 07:09:01,461 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 300000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-cars-config_testAddShardReplica, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 07:09:01,461 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-3-shard-cars-config_testAddShardReplica, persistent : true 07:09:01,462 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testAddShardReplica with journal batch size 1 07:09:01,462 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#2043651526] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:01,462 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#1173257754 created and ready for shard:member-3-shard-cars-config_testAddShardReplica 07:09:01,462 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#1173257754] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:01,462 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/shard-member-3:snapshot-read#-2125662849] 07:09:01,462 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/shard-member-3:snapshot-read#-2125662849] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:01,468 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.SnapshotOffer from Actor[pekko://cluster-test/system/in-memory-snapshot-store#-1775949552] 07:09:01,468 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: SnapshotOffer called. 07:09:01,468 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from -1 to 0 07:09:01,468 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Applying recovered snapshot 07:09:01,475 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 07:09:01,477 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-config_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#2142208793] with name member-3-shard-cars-config_testAddShardReplica-DTCL-publisher 07:09:01,477 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-3-shard-cars-config_testAddShardReplica: state snapshot applied in 2.292 ms 07:09:01,477 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#2142208793] 07:09:01,477 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING]] 07:09:01,477 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Recovery snapshot applied for member-3-shard-cars-config_testAddShardReplica in 9.303 ms: snapshotIndex=0, snapshotTerm=1, journal-size=0 07:09:01,478 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 07:09:01,478 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-config_testAddShardReplica-DTCL-publisher: 159.4 μs 07:09:01,478 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#2142208793] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:01,478 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:09:01,478 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Received ReplicatedLogEntry for recovery: index: 1, size: 267 07:09:01,478 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING]] 07:09:01,478 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[pekko://cluster-test/deadLetters] 07:09:01,478 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Received apply journal entries for recovery, applying to state: 1 to 1 07:09:01,478 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from 0 to 1 07:09:01,478 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:09:01,478 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Received ReplicatedLogEntry for recovery: index: 2, size: 235 07:09:01,478 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[pekko://cluster-test/deadLetters] 07:09:01,478 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Received apply journal entries for recovery, applying to state: 2 to 2 07:09:01,478 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: starting log recovery batch with max size 1 07:09:01,485 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@39219786}, purged=MutableUnsignedLongSet{size=0}} 07:09:01,486 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 07:09:01,486 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 07:09:01,486 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from 1 to 2 07:09:01,486 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:09:01,486 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Received ReplicatedLogEntry for recovery: index: 3, size: 56 07:09:01,486 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[pekko://cluster-test/deadLetters] 07:09:01,486 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Received apply journal entries for recovery, applying to state: 3 to 3 07:09:01,486 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: starting log recovery batch with max size 1 07:09:01,486 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 07:09:01,486 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 07:09:01,486 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from 2 to 3 07:09:01,486 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:01,486 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Recovery completed in 18.16 ms - Switching actor to Follower - last log index = 3, last log term = 1, snapshot index = 0, snapshot term = 1, journal size = 3 07:09:01,486 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 07:09:01,486 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized 07:09:01,486 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized - notifying 1 OnShardInitialized callbacks 07:09:01,487 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#2043651526] 07:09:01,487 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#2043651526] 07:09:01,487 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:09:01,487 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:01,487 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , received role change from null to Follower 07:09:01,487 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1915835000] 07:09:01,487 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 07:09:01,488 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 18000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-cars-config_testAddShardReplica, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 07:09:01,488 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplica 07:09:01,488 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testAddShardReplica from null to Follower 07:09:01,488 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized - notifying 1 OnShardInitialized callbacks 07:09:02,399 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-7086739839258536791] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 07:09:02,403 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2557] ignored due to non-matching UID, quarantine requested for [-7086739839258536791] but current is [-7333825138389519978]. Reason: ActorSystem terminated 07:09:02,473 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 07:09:02,473 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 07:09:02,511 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2558/system/clusterReceptionist/replicator#2142557313]] intended for system uid [-7086739839258536791], self uid is [-7333825138389519978] 07:09:02,511 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-654813042] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:02,511 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 07:09:02,512 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-654813042] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:02,512 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testAddShardReplica, leaderId=member-1-shard-cars-config_testAddShardReplica, leaderPayloadVersion=13] 07:09:02,512 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized - notifying 1 OnShardInitialized callbacks 07:09:02,512 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-654813042] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:02,513 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done false 07:09:02,514 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done true 07:09:02,625 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-654813042] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:02,926 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - New leader address: pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 07:09:02,926 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 07:09:02,946 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-654813042] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:03,461 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 07:09:03,610 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2559/system/clusterReceptionist/replicator#840305192]] intended for system uid [-7086739839258536791], self uid is [-7333825138389519978] 07:09:04,481 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 07:09:05,455 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2558/system/clusterReceptionist/replicator#2142557313]] intended for system uid [-7086739839258536791], self uid is [-7333825138389519978] 07:09:05,501 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 07:09:05,544 AM [cluster-test-pekko.actor.default-dispatcher-18] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Down)]. 07:09:05,545 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:05,545 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:05,550 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:05,550 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:05,631 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2559/system/clusterReceptionist/replicator#840305192]] intended for system uid [-7086739839258536791], self uid is [-7333825138389519978] 07:09:06,520 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 07:09:06,564 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is removing unreachable node [pekko://cluster-test@127.0.0.1:2557] 07:09:06,565 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberRemoved: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:06,565 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberRemoved: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:06,565 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2557] ignored due to non-matching UID, quarantine requested for [-7086739839258536791] but current is [-7333825138389519978]. Reason: Cluster member removed, previous status [Down] 07:09:06,569 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberRemoved: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:06,569 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received MemberRemoved: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:06,569 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2557] ignored due to non-matching UID, quarantine requested for [-7086739839258536791] but current is [-7333825138389519978]. Reason: Cluster member removed, previous status [Down] 07:09:07,541 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 07:09:07,543 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 07:09:07,545 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:07,545 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 07:09:07,545 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 07:09:07,545 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is ready - notifying 1 OnShardInitialized callbacks 07:09:07,545 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 07:09:07,545 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 07:09:07,545 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 07:09:07,545 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:09:07,545 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 07:09:07,545 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 07:09:07,546 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 07:09:07,546 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 07:09:07,546 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 07:09:07,548 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: retiring state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, nanosAgo=6764645106, purgedHistories=MutableUnsignedLongSet{size=0}}, outdated by request from client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} 07:09:07,552 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, nanosAgo=18680, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} 07:09:07,553 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-2097920714], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 07:09:07,553 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-2097920714], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 07:09:07,553 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-2097920714], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 486.0 μs 07:09:07,554 AM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: allocating new open snapshot member-3-datastore-config_testAddShardReplica-fe-1-txn-0-1 07:09:07,558 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} 07:09:07,558 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testAddShardReplica 07:09:07,558 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 07:09:07,558 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onShutDown 07:09:07,558 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testAddShardReplica 07:09:07,559 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 07:09:07,559 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 07:09:07,560 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:09:07,560 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:09:07,565 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:07,565 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:07,565 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:07,566 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:07,567 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 07:09:07,580 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} 07:09:07,580 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testAddShardReplica 07:09:07,581 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 1 shards 07:09:07,581 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: onShutDown 07:09:07,581 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testAddShardReplica 07:09:07,582 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 07:09:07,582 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 07:09:07,582 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} 07:09:07,582 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testAddShardReplica 07:09:07,582 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 1 shards 07:09:07,583 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: onShutDown 07:09:07,583 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testAddShardReplica 07:09:07,583 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 07:09:07,583 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 07:09:07,585 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:09:07,585 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:09:08,574 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-7333825138389519978] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 07:09:08,603 AM [cluster-test-pekko.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:08,604 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [2913752259598775779] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 07:09:08,604 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 07:09:08,604 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:08,605 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 07:09:08,605 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 07:09:08,605 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 07:09:08,605 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 07:09:08,605 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:08,605 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 07:09:08,605 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 07:09:08,605 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 07:09:08,606 AM [cluster-test-pekko.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:08,607 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 07:09:08,610 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [01/31/2025 07:09:08.613] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:09:08,631 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplica, generation=0} 07:09:08,632 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testAddShardReplica 07:09:08,632 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 07:09:08,632 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: onShutDown 07:09:08,632 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Initiating leader transfer 07:09:08,633 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testAddShardReplica, leaderId=null, leaderPayloadVersion=13] 07:09:08,633 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@299d162d 07:09:08,633 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@299d162d 07:09:08,633 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddShardReplica: pauseLeader successfully completed - doing transfer 07:09:08,633 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Attempting to transfer leadership 07:09:08,633 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:09:08,633 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:09:09,624 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 07:09:12,685 AM [cluster-test-pekko.actor.default-dispatcher-18] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2559, Up)]. 07:09:12,686 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:09:12,686 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:09:14,725 AM [cluster-test-pekko.actor.default-dispatcher-18] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Up)]. 07:09:14,725 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 07:09:14,725 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:14,726 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:15,410 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 07:09:16,50 AM [cluster-test-pekko.actor.default-dispatcher-19] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 07:09:17,643 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddShardReplica: leader transfer aborted 07:09:17,644 AM [cluster-test-shard-dispatcher-31] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddShardReplica: Failed to transfer leadership in 9.011 s 07:09:17,644 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: In unpauseLeader 07:09:17,646 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: leader transfer failed - sending PoisonPill 07:09:17,646 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Leadership transfer expired 07:09:17,647 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testAddShardReplica 07:09:17,648 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 07:09:17,648 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 07:09:17,648 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplica, generation=0} 07:09:17,649 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testAddShardReplica 07:09:17,649 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 1 shards 07:09:17,649 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: onShutDown 07:09:17,649 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Initiating leader transfer 07:09:17,649 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testAddShardReplica, leaderId=null, leaderPayloadVersion=13] 07:09:17,649 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7c7f6a5b 07:09:17,649 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7c7f6a5b 07:09:17,649 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testAddShardReplica: pauseLeader successfully completed - doing transfer 07:09:17,649 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Attempting to transfer leadership 07:09:17,649 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:09:17,649 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:09:17,984 AM [cluster-test-shard-dispatcher-31] [WARN] Shard - member-1-shard-cars-oper_testAddShardReplica: At least 1 followers need to be active, Switching member-1-shard-cars-oper_testAddShardReplica from Leader to IsolatedLeader 07:09:17,984 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-cars-oper_testAddShardReplica, member-2-shard-cars-oper_testAddShardReplica] 07:09:17,984 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:09:17,984 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 07:09:17,984 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 07:09:17,984 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testAddShardReplica: leader transfer aborted 07:09:17,984 AM [cluster-test-shard-dispatcher-31] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testAddShardReplica: Failed to transfer leadership in 335.4 ms 07:09:17,985 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: In unpauseLeader 07:09:17,985 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: leader transfer failed - sending PoisonPill 07:09:17,985 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 07:09:17,985 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from Leader to IsolatedLeader 07:09:17,985 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddShardReplica from Leader to IsolatedLeader 07:09:17,985 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testAddShardReplica 07:09:17,986 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 07:09:17,986 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 07:09:17,988 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:09:17,988 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:09:17,990 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [01/31/2025 07:09:17.993] [cluster-test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:09:18,24 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 07:09:18,34 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-6100457975772501849] 07:09:18,34 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 07:09:18,36 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 07:09:18,36 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 07:09:18,39 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:09:18,39 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:09:18,45 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:18,46 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:18,47 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForAllShards 07:09:18,49 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 07:09:18,49 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 07:09:18,49 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 07:09:18,52 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 07:09:18,56 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-79064321] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:18,57 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 07:09:18,58 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:18,58 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:18,58 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:18,58 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:18,58 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:18,82 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 07:09:18,82 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@4b03ed5d} 07:09:18,82 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} started 07:09:18,83 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:18,83 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:18,83 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:18,84 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:18,84 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:18,85 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:18,85 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards on shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#622346733] 07:09:18,85 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:18,85 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:18,85 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:18,85 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards on shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#1601808150] 07:09:18,86 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, persistent : true 07:09:18,86 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForAllShards 07:09:18,87 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with journal batch size 1 07:09:18,87 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, persistent : true 07:09:18,87 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards/shard-member-1:snapshot-read#-1792096551] 07:09:18,87 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#-124804811 created and ready for shard:member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:18,87 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards/shard-member-1:snapshot-read#-2022736225] 07:09:18,87 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#384733248 created and ready for shard:member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:18,87 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with journal batch size 1 07:09:18,88 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 07:09:18,88 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#622346733] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:18,88 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#1601808150] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:18,89 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#-124804811] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:18,89 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards/shard-member-1:snapshot-read#-1792096551] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:18,89 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#384733248] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:18,89 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards/shard-member-1:snapshot-read#-2022736225] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:18,89 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-55372383] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:18,91 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 07:09:18,91 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:18,91 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:18,92 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:18,92 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:18,92 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:18,92 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:18,92 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:18,92 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 07:09:18,92 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards is initialized 07:09:18,92 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 07:09:18,92 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:18,92 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 07:09:18,92 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:18,92 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:18,92 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 07:09:18,92 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards is initialized 07:09:18,92 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 07:09:18,93 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards from null to Follower 07:09:18,93 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 07:09:18,93 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:18,93 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 07:09:18,93 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 07:09:18,93 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards from null to Follower 07:09:18,94 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 07:09:18,95 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@1e503480} 07:09:18,95 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} started 07:09:18,97 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:18,97 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:18,97 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:18,97 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:18,97 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-1090053166] 07:09:18,97 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:18,104 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, persistent : true 07:09:18,104 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-1090053166] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:18,104 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards/shard-member-1:snapshot-read#235360223] 07:09:18,104 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#-774811396 created and ready for shard:member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:18,104 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with journal batch size 1 07:09:18,104 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards/shard-member-1:snapshot-read#235360223] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:18,105 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:18,105 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:18,105 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 07:09:18,105 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:18,105 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 07:09:18,111 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:18,111 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:18,111 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#1748010063] 07:09:18,111 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 07:09:18,111 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards is initialized 07:09:18,111 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 07:09:18,112 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, persistent : true 07:09:18,112 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#-1188416530 created and ready for shard:member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:18,112 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards/shard-member-1:snapshot-read#-99421328] 07:09:18,114 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards from null to Follower 07:09:18,114 AM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-3-datastore-config_testAddShardReplica-fe-0-txn-0-0} 07:09:18,114 AM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-0} 07:09:18,114 AM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-3-datastore-config_testAddShardReplica-fe-1-txn-0-0} 07:09:18,115 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with journal batch size 1 07:09:18,115 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:09:18,119 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:18,119 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:18,119 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 07:09:18,119 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards is initialized 07:09:18,119 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 07:09:18,119 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:18,119 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 07:09:18,119 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 07:09:18,119 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards from null to Follower 07:09:18,126 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [7044869482507858664] 07:09:18,126 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 07:09:18,128 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:09:18,128 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 07:09:18,131 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:09:18,131 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:09:18,139 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:18,140 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:18,142 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForAllShards 07:09:18,149 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 07:09:18,150 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#2138724628] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:18,151 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 07:09:18,151 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:18,152 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:18,156 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 07:09:18,157 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@29be9aa1} 07:09:18,157 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} started 07:09:18,158 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:18,158 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:18,158 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:18,158 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:18,159 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, persistent : true 07:09:18,159 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, persistent : true 07:09:18,159 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with journal batch size 1 07:09:18,159 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with journal batch size 1 07:09:18,159 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:18,159 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#-1390153511 created and ready for shard:member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:18,159 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards/shard-member-2:snapshot-read#134643975] 07:09:18,159 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#-1470678399 created and ready for shard:member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:18,159 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards/shard-member-2:snapshot-read#-984943567] 07:09:18,159 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#-1223654531] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:18,160 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards#-1646388863] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:18,160 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#-1390153511] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:18,160 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#-1470678399] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:18,160 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards/shard-member-2:snapshot-read#-984943567] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:18,160 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards/shard-member-2:snapshot-read#134643975] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:18,161 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForAllShards 07:09:18,162 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:18,163 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:18,163 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 07:09:18,163 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 07:09:18,163 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards is initialized 07:09:18,163 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:18,163 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:18,163 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#1983098687] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:18,167 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 07:09:18,167 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards is initialized 07:09:18,168 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 07:09:18,168 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:18,168 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 07:09:18,168 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 07:09:18,168 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards from null to Follower 07:09:18,168 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 07:09:18,168 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:18,170 AM [cluster-test-pekko.actor.default-dispatcher-33] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 07:09:18,170 AM [cluster-test-pekko.actor.default-dispatcher-33] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 07:09:18,170 AM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards from null to Follower 07:09:18,170 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 07:09:18,170 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:18,171 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:18,171 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 07:09:18,172 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@147ec900} 07:09:18,172 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} started 07:09:18,172 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:18,172 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:18,172 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:18,173 AM [cluster-test-shard-dispatcher-36] [INFO] Shard - Shard created : member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, persistent : true 07:09:18,173 AM [cluster-test-shard-dispatcher-36] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with journal batch size 1 07:09:18,173 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#1030299821] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:18,173 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards/shard-member-2:snapshot-read#1805223309] 07:09:18,173 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#1718858158 created and ready for shard:member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:18,174 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#1718858158] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:18,174 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - Shard created : member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, persistent : true 07:09:18,174 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#-1770849802 created and ready for shard:member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:18,174 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards/shard-member-2:snapshot-read#321675390] 07:09:18,174 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with journal batch size 1 07:09:18,175 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:18,175 AM [cluster-test-shard-dispatcher-36] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:18,175 AM [cluster-test-shard-dispatcher-36] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 07:09:18,175 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:18,175 AM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 07:09:18,175 AM [cluster-test-shard-dispatcher-36] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards is initialized 07:09:18,175 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:18,175 AM [cluster-test-shard-dispatcher-36] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:18,175 AM [cluster-test-shard-dispatcher-36] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 07:09:18,175 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:18,175 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 07:09:18,175 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards is initialized 07:09:18,175 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 07:09:18,175 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 07:09:18,175 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 07:09:18,175 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 07:09:18,175 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards from null to Follower 07:09:18,175 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards from null to Follower 07:09:18,180 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:09:18,182 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 07:09:18,183 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 07:09:18,184 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:18,184 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:18,184 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:18,184 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:18,184 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:18,184 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:18,184 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:18,184 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:18,184 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:18,184 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:18,184 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:18,184 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:18,184 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:18,184 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:18,195 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [5611544438788780289] 07:09:18,195 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 07:09:18,198 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:09:18,198 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 07:09:18,202 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:09:18,202 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:09:18,208 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:18,210 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:18,212 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForAllShards 07:09:18,219 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 07:09:18,219 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1563492467] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:18,226 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 07:09:18,226 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:18,226 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:18,230 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 07:09:18,231 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@53ab41b} 07:09:18,232 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} started 07:09:18,232 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:18,232 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:18,232 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:18,233 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:18,233 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, persistent : true 07:09:18,234 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:18,234 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with journal batch size 1 07:09:18,234 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, persistent : true 07:09:18,236 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with journal batch size 1 07:09:18,236 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#-1572517700 created and ready for shard:member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:18,236 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#-1243059076 created and ready for shard:member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:18,236 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards/shard-member-3:snapshot-read#1962211515] 07:09:18,236 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#1874461362] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:18,239 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:18,239 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:18,239 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForAllShards 07:09:18,240 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 07:09:18,241 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards/shard-member-3:snapshot-read#686769264] 07:09:18,243 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 07:09:18,245 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards#931594695] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:18,245 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#-1572517700] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:18,245 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#-1243059076] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:18,245 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards/shard-member-3:snapshot-read#1962211515] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:18,245 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards/shard-member-3:snapshot-read#686769264] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:18,249 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:18,249 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 07:09:18,249 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards is initialized 07:09:18,249 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:18,249 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:18,249 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:18,249 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:18,249 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:18,249 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:18,249 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 07:09:18,249 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards is initialized 07:09:18,250 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 07:09:18,250 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:18,250 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:18,250 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 07:09:18,250 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-351800804] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:18,250 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 07:09:18,250 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards from null to Follower 07:09:18,250 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 07:09:18,250 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:18,250 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 07:09:18,251 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:18,251 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 07:09:18,251 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards from null to Follower 07:09:18,251 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 07:09:18,254 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 07:09:18,255 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:18,255 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:18,255 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:18,255 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:18,255 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:18,258 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 07:09:18,258 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} started 07:09:18,259 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@35806fa4} 07:09:18,259 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 07:09:18,259 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:18,259 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:18,259 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:18,259 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:18,259 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#395789648] 07:09:18,259 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:18,260 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Shard created : member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, persistent : true 07:09:18,260 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:18,260 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:18,260 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#1337684557] 07:09:18,260 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Shard created : member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, persistent : true 07:09:18,260 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with journal batch size 1 07:09:18,260 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#-1512794898 created and ready for shard:member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:18,261 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards/shard-member-3:snapshot-read#-1161572007] 07:09:18,261 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#395789648] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:18,261 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#1337684557] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:18,261 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards/shard-member-3:snapshot-read#-1994621904] 07:09:18,261 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#-1326409616 created and ready for shard:member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:18,261 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with journal batch size 1 07:09:18,262 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:18,262 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:18,262 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:18,262 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:18,262 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 07:09:18,262 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:18,262 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 07:09:18,262 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 07:09:18,262 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:18,262 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 07:09:18,262 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:18,262 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:18,262 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 07:09:18,262 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards is initialized 07:09:18,262 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 07:09:18,262 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards is initialized 07:09:18,263 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 07:09:18,263 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 07:09:18,263 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards from null to Follower 07:09:18,263 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards from null to Follower 07:09:18,413 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 07:09:18,413 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Received ElectionTimeout - switching to Candidate 07:09:18,414 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards] 07:09:18,414 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 07:09:18,414 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:09:18,414 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 07:09:18,414 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:18,414 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 07:09:18,414 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards from Follower to Candidate 07:09:18,454 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 07:09:18,454 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Received ElectionTimeout - switching to Candidate 07:09:18,454 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Election: Candidate has following voting peers: [member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards] 07:09:18,454 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 07:09:18,454 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:09:18,454 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 07:09:18,454 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:18,454 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 07:09:18,455 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards from Follower to Candidate 07:09:18,483 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 07:09:18,483 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Received ElectionTimeout - switching to Candidate 07:09:18,484 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards] 07:09:18,484 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 07:09:18,484 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:09:18,484 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 07:09:18,484 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:18,484 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 07:09:18,484 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards from Follower to Candidate 07:09:18,494 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 07:09:18,494 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Received ElectionTimeout - switching to Candidate 07:09:18,494 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards] 07:09:18,494 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 07:09:18,494 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:09:18,494 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 07:09:18,494 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:18,494 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 07:09:18,494 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards from Follower to Candidate 07:09:18,814 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 07:09:18,814 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 07:09:18,824 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 07:09:18,824 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 07:09:18,824 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 07:09:18,824 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 07:09:18,854 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 07:09:18,854 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 07:09:19,54 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 07:09:19,54 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 07:09:19,55 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:19,55 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:19,55 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:19,55 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:19,55 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:19,55 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:09:19,55 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:19,55 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:19,55 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:19,55 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:19,55 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:19,55 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:19,55 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:19,55 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:19,55 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:19,55 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:19,55 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:19,55 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:19,55 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:19,55 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:09:19,55 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:19,55 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:19,55 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:19,55 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:19,55 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:19,55 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:19,55 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:19,56 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:19,61 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:19,61 AM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:19,61 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:19,61 AM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:19,61 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:19,62 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:09:19,62 AM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:19,62 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:19,62 AM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:19,62 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:19,62 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:19,62 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:19,62 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:19,62 AM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:19,62 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:19,62 AM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:19,62 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:19,62 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:09:19,62 AM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:19,62 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:19,62 AM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:19,62 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:19,62 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:19,62 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:19,143 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 07:09:19,144 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 07:09:19,144 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1} to peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:19,144 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1} to peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:19,145 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:19,145 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:19,146 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 07:09:19,146 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:09:19,146 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 07:09:19,146 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:09:19,146 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:09:19,146 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:09:19,146 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:09:19,146 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:09:19,146 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:09:19,146 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:09:19,147 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): RaftRPC message received RequestVoteReply{term=3, voteGranted=true}, my term is 3 07:09:19,148 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): handleRequestVoteReply: RequestVoteReply{term=3, voteGranted=true}, current voteCount: 1 07:09:19,148 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Election: Leader has following peers: [member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards] 07:09:19,148 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:19,148 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:19,148 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 07:09:19,148 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 07:09:19,148 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 07:09:19,148 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 07:09:19,148 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 07:09:19,148 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards from Candidate to Leader 07:09:19,149 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done false 07:09:19,150 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 07:09:19,150 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done false 07:09:19,150 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 07:09:19,150 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:19,150 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 07:09:19,155 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 07:09:19,155 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:19,167 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:19,167 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:19,167 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:19,167 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:19,167 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:19,167 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:09:19,167 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:19,167 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:19,167 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:19,167 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:19,167 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:19,167 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:19,167 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:19,167 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:19,167 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:19,167 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:19,167 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:19,167 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:09:19,167 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:19,167 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:19,167 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:19,167 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:19,167 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:19,167 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:19,184 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 07:09:19,184 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 07:09:19,184 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1} to peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:19,184 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1} to peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:19,185 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:19,185 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 07:09:19,185 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:09:19,185 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:09:19,185 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:09:19,186 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:09:19,187 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): RaftRPC message received RequestVoteReply{term=3, voteGranted=true}, my term is 3 07:09:19,187 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): handleRequestVoteReply: RequestVoteReply{term=3, voteGranted=true}, current voteCount: 1 07:09:19,187 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Election: Leader has following peers: [member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards] 07:09:19,187 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:19,187 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:19,187 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 07:09:19,187 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:09:19,187 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:09:19,187 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:19,187 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 07:09:19,187 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 07:09:19,187 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:09:19,187 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 07:09:19,187 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:09:19,187 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 07:09:19,187 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 07:09:19,187 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards from Candidate to Leader 07:09:19,187 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testChangeMemberVotingStatesForAllShards is ready 07:09:19,188 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done false 07:09:19,188 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 07:09:19,188 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testChangeMemberVotingStatesForAllShards is ready 07:09:19,188 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done false 07:09:19,189 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 07:09:19,189 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:19,189 AM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 07:09:19,190 AM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testChangeMemberVotingStatesForAllShards is ready 07:09:19,190 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 07:09:19,190 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:19,216 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 07:09:19,223 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 07:09:19,223 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 07:09:19,223 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1} to peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:19,224 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1} to peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:19,225 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:19,225 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 07:09:19,225 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:09:19,225 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:09:19,225 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:09:19,225 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:09:19,225 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:19,225 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 07:09:19,225 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:09:19,225 AM [cluster-test-shard-dispatcher-36] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:09:19,225 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:09:19,226 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:09:19,226 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): RaftRPC message received RequestVoteReply{term=3, voteGranted=true}, my term is 3 07:09:19,226 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): handleRequestVoteReply: RequestVoteReply{term=3, voteGranted=true}, current voteCount: 1 07:09:19,227 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards] 07:09:19,227 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:19,227 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:19,227 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 07:09:19,227 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 07:09:19,227 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 07:09:19,227 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 07:09:19,227 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 07:09:19,227 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards from Candidate to Leader 07:09:19,228 AM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done false 07:09:19,228 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 07:09:19,228 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done false 07:09:19,228 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 07:09:19,229 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 07:09:19,229 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:19,229 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 07:09:19,229 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:19,274 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 07:09:19,274 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 07:09:19,274 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1} to peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:19,274 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1} to peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:19,275 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:19,275 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 07:09:19,275 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:09:19,275 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:09:19,275 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:09:19,275 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:09:19,276 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:19,276 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 07:09:19,276 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:09:19,276 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:09:19,276 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:09:19,276 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:09:19,277 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): RaftRPC message received RequestVoteReply{term=3, voteGranted=true}, my term is 3 07:09:19,277 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): handleRequestVoteReply: RequestVoteReply{term=3, voteGranted=true}, current voteCount: 1 07:09:19,277 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Election: Leader has following peers: [member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards] 07:09:19,277 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:19,277 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:19,277 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 07:09:19,277 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 07:09:19,277 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 07:09:19,277 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 07:09:19,278 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 07:09:19,278 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards from Candidate to Leader 07:09:19,278 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testChangeMemberVotingStatesForAllShards is ready 07:09:19,278 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 07:09:19,278 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 07:09:19,278 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 07:09:19,278 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 07:09:19,278 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done false 07:09:19,278 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done false 07:09:19,278 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 07:09:19,278 AM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 07:09:19,278 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testChangeMemberVotingStatesForAllShards is ready 07:09:19,278 AM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testChangeMemberVotingStatesForAllShards is ready 07:09:19,278 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 07:09:19,279 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 07:09:19,279 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 07:09:19,279 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-79064321] 07:09:19,279 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:09:19,279 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#622346733] 07:09:19,279 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#622346733] 07:09:19,279 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 07:09:19,279 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:19,279 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 07:09:19,279 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:19,280 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#2138724628] 07:09:19,280 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 07:09:19,280 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#-1223654531] 07:09:19,280 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#-1223654531] 07:09:19,280 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1563492467] 07:09:19,280 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 07:09:19,280 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#1874461362] 07:09:19,281 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#1874461362] 07:09:19,286 AM [main] [INFO] ClusterAdminRpcService - Change member voting states for all shards 07:09:19,286 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testChangeMemberVotingStatesForAllShards 07:09:19,287 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-2=false, member-3=false}] 07:09:19,287 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 07:09:19,288 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards=false, member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:19,288 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=people, meberVotingStatusMap={member-2=false, member-3=false}] 07:09:19,288 AM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards=false, member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 07:09:19,288 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 07:09:19,288 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testChangeMemberVotingStatesForAllShards 07:09:19,288 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-2=false, member-3=false}] 07:09:19,288 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards peerAddress = pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:19,288 AM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Not leader - forwarding to leader ActorSelection[Anchor(pekko://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards)] 07:09:19,288 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 07:09:19,288 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=people, meberVotingStatusMap={member-2=false, member-3=false}] 07:09:19,288 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:19,289 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 07:09:19,289 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 07:09:19,289 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards peerAddress = pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:19,289 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:19,289 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Not leader - forwarding to leader ActorSelection[Anchor(pekko://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards)] 07:09:19,289 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 07:09:19,289 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards peerAddress = pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:19,289 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Not leader - forwarding to leader ActorSelection[Anchor(pekko://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards)] 07:09:19,289 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:19,289 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 07:09:19,289 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-people-config_testChangeMemberVotingStatesForAllShards peerAddress = pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:19,289 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Not leader - forwarding to leader ActorSelection[Anchor(pekko://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards)] 07:09:19,290 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 07:09:19,290 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 07:09:19,290 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING]] 07:09:19,290 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]] 07:09:19,290 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 07:09:19,290 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 07:09:19,290 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=9a63d9da-618c-4aad-aeae-110f31ac4e15}, logIndex: 0, isSendImmediate: true 07:09:19,290 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Setting last applied to 0 07:09:19,290 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 07:09:19,290 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 07:09:19,290 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:19,290 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:19,290 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 07:09:19,290 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: AppendEntries{term=3, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]} 07:09:19,290 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 07:09:19,290 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:19,290 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:19,290 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: AppendEntries{term=3, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]} 07:09:19,291 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING]] 07:09:19,291 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] 07:09:19,290 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards=false, member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 07:09:19,291 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]] 07:09:19,291 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 07:09:19,291 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 07:09:19,291 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 07:09:19,291 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 07:09:19,291 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 07:09:19,291 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING]] 07:09:19,291 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: New server configuration : [ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]] 07:09:19,291 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 07:09:19,291 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 07:09:19,291 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=68090474-a3eb-4125-ac13-2d9103c0ec28}, logIndex: 0, isSendImmediate: true 07:09:19,291 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=86e6e91d-92a6-4d9a-b9f4-7d1a48b3cadb}, logIndex: 0, isSendImmediate: true 07:09:19,291 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Setting last applied to 0 07:09:19,291 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 07:09:19,291 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Setting last applied to 0 07:09:19,291 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 07:09:19,291 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 07:09:19,291 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 07:09:19,291 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 07:09:19,291 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:19,291 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:19,291 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:19,291 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:19,291 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: AppendEntries{term=3, leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]} 07:09:19,291 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: AppendEntries{term=3, leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]} 07:09:19,291 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:19,291 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:19,291 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: AppendEntries{term=3, leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]} 07:09:19,291 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:19,291 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:19,291 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards=false, member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] 07:09:19,291 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: AppendEntries{term=3, leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]} 07:09:19,292 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 07:09:19,292 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 07:09:19,292 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] 07:09:19,293 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 07:09:19,293 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 07:09:19,293 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 07:09:19,293 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 07:09:19,294 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 07:09:19,294 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 07:09:19,294 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING]] 07:09:19,294 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]} 07:09:19,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 07:09:19,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 07:09:19,295 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]] 07:09:19,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 07:09:19,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 07:09:19,295 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 07:09:19,295 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 07:09:19,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING]] 07:09:19,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 07:09:19,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 07:09:19,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:19,295 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=461cdf7b-c0d6-44d8-a974-04e66e33f443}, logIndex: 0, isSendImmediate: true 07:09:19,295 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Setting last applied to 0 07:09:19,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 07:09:19,295 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 07:09:19,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 07:09:19,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 07:09:19,295 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 07:09:19,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 07:09:19,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]} 07:09:19,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 07:09:19,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 07:09:19,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 07:09:19,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 07:09:19,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, votingState=VOTING]] 07:09:19,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 07:09:19,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 07:09:19,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:19,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 07:09:19,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 07:09:19,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 07:09:19,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 07:09:19,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]} 07:09:19,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 07:09:19,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 07:09:19,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 07:09:19,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 07:09:19,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, votingState=VOTING]] 07:09:19,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 07:09:19,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 07:09:19,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:19,296 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 07:09:19,296 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 07:09:19,296 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 07:09:19,296 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 07:09:19,296 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 07:09:19,296 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 07:09:19,296 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 07:09:19,296 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 07:09:19,296 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done true 07:09:19,296 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done true 07:09:19,296 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:19,296 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:19,296 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: AppendEntries{term=3, leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]} 07:09:19,296 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:19,296 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:19,296 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: AppendEntries{term=3, leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]} 07:09:19,296 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] 07:09:19,297 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 07:09:19,297 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]} 07:09:19,297 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 07:09:19,297 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 07:09:19,297 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 07:09:19,297 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]} 07:09:19,297 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 07:09:19,297 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 07:09:19,297 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 07:09:19,297 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 07:09:19,297 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 07:09:19,297 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 07:09:19,297 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, votingState=VOTING]] 07:09:19,297 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 07:09:19,297 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 07:09:19,297 AM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done true 07:09:19,297 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:19,297 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 07:09:19,297 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, votingState=VOTING], PeerInfo [id=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING]] 07:09:19,297 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 07:09:19,297 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 07:09:19,297 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 07:09:19,297 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards} from local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:19,297 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 07:09:19,297 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 07:09:19,297 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 07:09:19,297 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:19,297 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]} 07:09:19,297 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 07:09:19,297 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 07:09:19,297 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 07:09:19,297 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 07:09:19,297 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 07:09:19,297 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 07:09:19,297 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 07:09:19,297 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards} from local shard pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:19,297 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 07:09:19,297 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard people 07:09:19,297 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 07:09:19,297 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 07:09:19,297 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 07:09:19,297 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 07:09:19,297 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards} from local shard pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:19,297 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard cars 07:09:19,297 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 07:09:19,297 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 07:09:19,297 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 07:09:19,297 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 07:09:19,297 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, votingState=VOTING]] 07:09:19,297 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 07:09:19,297 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 07:09:19,297 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:19,297 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 07:09:19,297 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 07:09:19,297 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 07:09:19,297 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 07:09:19,297 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 07:09:19,298 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 07:09:19,298 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 07:09:19,298 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done true 07:09:19,298 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 07:09:19,298 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 07:09:19,298 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 07:09:19,298 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done true 07:09:19,298 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 07:09:19,298 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done true 07:09:19,299 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]} 07:09:19,299 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 07:09:19,299 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 07:09:19,299 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 07:09:19,299 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 07:09:19,299 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, votingState=VOTING]] 07:09:19,299 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 07:09:19,299 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 07:09:19,299 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:19,299 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 07:09:19,299 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 07:09:19,299 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 07:09:19,299 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 07:09:19,299 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 07:09:19,299 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:19,299 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 07:09:19,299 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:19,299 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 07:09:19,299 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:19,299 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards} from local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:19,299 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard people 07:09:19,299 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 07:09:19,299 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 07:09:19,299 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 07:09:19,299 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 07:09:19,299 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done true 07:09:19,300 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 07:09:19,300 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:19,300 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]} 07:09:19,300 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 07:09:19,300 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 07:09:19,300 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 07:09:19,300 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 07:09:19,300 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, votingState=VOTING]] 07:09:19,300 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 07:09:19,300 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 07:09:19,300 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:19,300 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 07:09:19,300 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 07:09:19,300 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 07:09:19,300 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 07:09:19,300 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 07:09:19,300 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 07:09:19,300 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:19,300 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 07:09:19,300 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done true 07:09:19,301 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 07:09:19,301 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:19,301 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-79064321] 07:09:19,301 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 07:09:19,301 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:09:19,301 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:19,301 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#622346733] 07:09:19,301 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#622346733] 07:09:19,301 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 07:09:19,301 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:19,301 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-79064321] 07:09:19,301 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 07:09:19,302 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#1601808150] 07:09:19,302 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#1601808150] 07:09:19,302 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-55372383] 07:09:19,302 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 07:09:19,302 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-1090053166] 07:09:19,302 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-1090053166] 07:09:19,302 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-55372383] 07:09:19,302 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 07:09:19,302 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#1748010063] 07:09:19,302 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#1748010063] 07:09:19,302 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#2138724628] 07:09:19,302 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 07:09:19,302 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#-1223654531] 07:09:19,302 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#-1223654531] 07:09:19,303 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#2138724628] 07:09:19,303 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 07:09:19,303 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards#-1646388863] 07:09:19,303 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards#-1646388863] 07:09:19,303 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#1983098687] 07:09:19,303 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 07:09:19,303 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#1030299821] 07:09:19,303 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#1030299821] 07:09:19,303 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#1983098687] 07:09:19,303 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 07:09:19,303 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards#1621299303] 07:09:19,303 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards#1621299303] 07:09:19,303 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1563492467] 07:09:19,303 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 07:09:19,303 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#1874461362] 07:09:19,303 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#1874461362] 07:09:19,303 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1563492467] 07:09:19,304 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 07:09:19,304 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards#931594695] 07:09:19,304 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards#931594695] 07:09:19,304 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-351800804] 07:09:19,304 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 07:09:19,304 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#395789648] 07:09:19,304 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#395789648] 07:09:19,304 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-351800804] 07:09:19,304 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 07:09:19,304 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#1337684557] 07:09:19,304 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#1337684557] 07:09:19,304 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 07:09:19,304 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:19,304 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:19,304 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 07:09:19,305 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onShutDown 07:09:19,305 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onShutDown 07:09:19,305 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:19,305 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:19,305 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 07:09:19,305 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 07:09:19,305 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 07:09:19,306 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:19,306 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:19,306 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 07:09:19,306 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onShutDown 07:09:19,306 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onShutDown 07:09:19,306 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:19,306 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:19,306 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 07:09:19,306 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 07:09:19,307 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:09:19,307 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:09:19,310 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:19,310 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:19,310 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:19,311 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:19,312 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 07:09:19,326 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 07:09:19,327 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:19,327 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:19,327 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 07:09:19,327 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: onShutDown 07:09:19,327 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: onShutDown 07:09:19,328 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:19,328 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:19,328 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 07:09:19,329 AM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 07:09:19,329 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 07:09:19,329 AM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:19,329 AM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:19,329 AM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 07:09:19,329 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onShutDown 07:09:19,330 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: onShutDown 07:09:19,330 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:19,330 AM [cluster-test-shard-dispatcher-36] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:19,330 AM [cluster-test-pekko.actor.default-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 07:09:19,330 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 07:09:19,332 AM [cluster-test-pekko.actor.default-dispatcher-33] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:09:19,332 AM [cluster-test-pekko.actor.default-dispatcher-33] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:09:20,74 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 07:09:20,324 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [5611544438788780289] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 07:09:20,344 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [7044869482507858664] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 07:09:20,352 AM [cluster-test-pekko.actor.default-dispatcher-33] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:20,352 AM [cluster-test-pekko.actor.default-dispatcher-33] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:20,353 AM [cluster-test-pekko.actor.default-dispatcher-33] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [01/31/2025 07:09:20.356] [cluster-test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:09:20,369 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 07:09:20,369 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:20,369 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:20,369 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 07:09:20,370 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onShutDown 07:09:20,370 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onShutDown 07:09:20,370 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Initiating leader transfer 07:09:20,370 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Initiating leader transfer 07:09:20,370 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2f693307 07:09:20,370 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2f693307 07:09:20,370 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4849c2d 07:09:20,370 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 07:09:20,370 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Attempting to transfer leadership 07:09:20,370 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4849c2d 07:09:20,370 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 07:09:20,370 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Attempting to transfer leadership 07:09:20,370 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:20,370 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:20,370 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:20,370 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:20,370 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderId=null, leaderPayloadVersion=13] 07:09:20,371 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderId=null, leaderPayloadVersion=13] 07:09:20,824 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: leader transfer aborted 07:09:20,824 AM [cluster-test-shard-dispatcher-20] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 454.2 ms 07:09:20,824 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: In unpauseLeader 07:09:20,824 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: leader transfer failed - sending PoisonPill 07:09:20,824 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Leadership transfer expired 07:09:20,826 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 07:09:20,873 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: leader transfer aborted 07:09:20,874 AM [cluster-test-shard-dispatcher-19] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 503.7 ms 07:09:20,874 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: In unpauseLeader 07:09:20,874 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: leader transfer failed - sending PoisonPill 07:09:20,874 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Leadership transfer expired 07:09:20,874 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - Stopping Shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 07:09:20,875 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 07:09:20,875 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 07:09:20,875 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 07:09:20,876 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:20,876 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:20,876 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 07:09:20,876 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onShutDown 07:09:20,876 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Initiating leader transfer 07:09:20,876 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onShutDown 07:09:20,876 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Initiating leader transfer 07:09:20,876 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderId=null, leaderPayloadVersion=13] 07:09:20,877 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderId=null, leaderPayloadVersion=13] 07:09:20,877 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@64ec61dc 07:09:20,877 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@64ec61dc 07:09:20,877 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 07:09:20,877 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Attempting to transfer leadership 07:09:20,877 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:20,877 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@78e365a2 07:09:20,877 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:20,877 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@78e365a2 07:09:20,877 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 07:09:20,877 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Attempting to transfer leadership 07:09:20,877 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:20,877 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:21,384 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: leader transfer aborted 07:09:21,384 AM [cluster-test-shard-dispatcher-20] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 507.8 ms 07:09:21,385 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: In unpauseLeader 07:09:21,386 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: leader transfer failed - sending PoisonPill 07:09:21,386 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Leadership transfer expired 07:09:21,390 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 07:09:21,423 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: leader transfer aborted 07:09:21,424 AM [cluster-test-shard-dispatcher-18] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 547.1 ms 07:09:21,424 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: In unpauseLeader 07:09:21,424 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: leader transfer failed - sending PoisonPill 07:09:21,424 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Leadership transfer expired 07:09:21,424 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 07:09:21,425 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 07:09:21,425 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 07:09:21,426 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:09:21,426 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:09:21,430 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [01/31/2025 07:09:21.432] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:09:21,461 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 07:09:21,469 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-5311831021746447532] 07:09:21,469 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 07:09:21,471 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 07:09:21,471 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 07:09:21,475 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:09:21,475 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:09:21,486 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:21,488 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:21,490 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testGetPrefixShardRole 07:09:21,492 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 07:09:21,492 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 07:09:21,492 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 07:09:21,493 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#-1337931814] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,494 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 07:09:21,495 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 07:09:21,495 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-default-config_testGetPrefixShardRole 07:09:21,496 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:21,496 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole 07:09:21,498 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testGetPrefixShardRole: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testGetPrefixShardRole, generation=0} 07:09:21,499 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@61cc9027} 07:09:21,499 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testGetPrefixShardRole, generation=0} started 07:09:21,499 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:21,500 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:21,505 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:21,507 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config_testGetPrefixShardRole 07:09:21,509 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole 07:09:21,509 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole to pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole 07:09:21,509 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-default-config_testGetPrefixShardRole on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole on shard member-1-shard-default-config_testGetPrefixShardRole whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole#1935649053] 07:09:21,510 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-default-config_testGetPrefixShardRole, persistent : true 07:09:21,511 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole#1935649053] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,511 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole-notifier#-781924658 created and ready for shard:member-1-shard-default-config_testGetPrefixShardRole 07:09:21,511 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-default-config_testGetPrefixShardRole with journal batch size 1 07:09:21,512 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole-notifier#-781924658] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,512 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole/shard-member-1:snapshot-read#-229560222] 07:09:21,512 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole/shard-member-1:snapshot-read#-229560222] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,513 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testGetPrefixShardRole 07:09:21,514 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole#-1407426375] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,514 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 07:09:21,515 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:21,515 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-default-config_testGetPrefixShardRole: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:21,516 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [default] 07:09:21,516 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-default-config_testGetPrefixShardRole is initialized 07:09:21,516 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-config_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 07:09:21,516 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:21,516 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole (Follower): Received TimeoutNow - switching to Candidate 07:09:21,516 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole (Candidate): Election: Candidate has following voting peers: [] 07:09:21,516 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-default-config_testGetPrefixShardRole (Candidate): Starting new election term 1 07:09:21,516 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-default-config_testGetPrefixShardRole (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:09:21,516 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-config_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 07:09:21,516 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:21,517 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 07:09:21,517 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-default-oper_testGetPrefixShardRole 07:09:21,517 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:21,517 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-oper_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole 07:09:21,518 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , received role change from null to Follower 07:09:21,518 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , received role change from Follower to Candidate 07:09:21,519 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole (Candidate): Received ElectionTimeout 07:09:21,519 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole (Leader): Election: Leader has following peers: [] 07:09:21,519 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-default-config_testGetPrefixShardRole (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 07:09:21,519 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-config_testGetPrefixShardRole: onLeadershipChange, isLeader: true, hasLeader : true 07:09:21,519 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: became leader with frontend state for [] 07:09:21,519 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , received role change from Candidate to Leader 07:09:21,519 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , registered listener pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole 07:09:21,519 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config_testGetPrefixShardRole, leaderId=member-1-shard-default-config_testGetPrefixShardRole, leaderPayloadVersion=13] 07:09:21,520 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-default-config_testGetPrefixShardRole from Candidate to Leader 07:09:21,520 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testGetPrefixShardRole is ready 07:09:21,520 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testGetPrefixShardRole: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testGetPrefixShardRole, generation=0} 07:09:21,520 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testGetPrefixShardRole, generation=0} started 07:09:21,520 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@6f0b726e} 07:09:21,521 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:21,521 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-default-oper_testGetPrefixShardRole 07:09:21,521 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-oper_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole 07:09:21,521 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-oper_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole to pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole 07:09:21,521 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-default-oper_testGetPrefixShardRole on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole on shard member-1-shard-default-oper_testGetPrefixShardRole whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole#12196570] 07:09:21,522 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-default-oper_testGetPrefixShardRole, persistent : true 07:09:21,522 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole#12196570] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,522 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole/shard-member-1:snapshot-read#1568593176] 07:09:21,522 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole-notifier#312726668 created and ready for shard:member-1-shard-default-oper_testGetPrefixShardRole 07:09:21,522 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole-notifier#312726668] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,522 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole/shard-member-1:snapshot-read#1568593176] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,522 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-default-oper_testGetPrefixShardRole with journal batch size 1 07:09:21,523 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:21,523 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-default-oper_testGetPrefixShardRole: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:21,523 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [default] 07:09:21,523 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-default-oper_testGetPrefixShardRole is initialized 07:09:21,523 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-oper_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 07:09:21,523 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:21,523 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , received role change from null to Follower 07:09:21,523 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Follower): Received TimeoutNow - switching to Candidate 07:09:21,523 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Candidate): Election: Candidate has following voting peers: [] 07:09:21,523 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Candidate): Starting new election term 1 07:09:21,523 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:09:21,523 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-oper_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 07:09:21,523 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:21,523 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , received role change from Follower to Candidate 07:09:21,527 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Candidate): Received ElectionTimeout 07:09:21,527 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Leader): Election: Leader has following peers: [] 07:09:21,527 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 07:09:21,527 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-oper_testGetPrefixShardRole: onLeadershipChange, isLeader: true, hasLeader : true 07:09:21,527 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: became leader with frontend state for [] 07:09:21,527 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , received role change from Candidate to Leader 07:09:21,527 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , registered listener pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole 07:09:21,527 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-oper_testGetPrefixShardRole, leaderId=member-1-shard-default-oper_testGetPrefixShardRole, leaderPayloadVersion=13] 07:09:21,527 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-default-oper_testGetPrefixShardRole from Candidate to Leader 07:09:21,527 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testGetPrefixShardRole is ready 07:09:21,571 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#-1337931814] 07:09:21,572 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : default 07:09:21,572 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole#1935649053] 07:09:21,572 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testGetPrefixShardRole, generation=0} 07:09:21,573 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-default-config_testGetPrefixShardRole 07:09:21,573 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 07:09:21,573 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: onShutDown 07:09:21,573 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@40ce03fd 07:09:21,573 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-default-config_testGetPrefixShardRole: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@40ce03fd 07:09:21,574 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-default-config_testGetPrefixShardRole 07:09:21,575 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 07:09:21,575 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 07:09:21,576 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testGetPrefixShardRole, generation=0} 07:09:21,576 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-default-oper_testGetPrefixShardRole 07:09:21,576 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 1 shards 07:09:21,576 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: onShutDown 07:09:21,576 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@4f76bc69 07:09:21,576 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-default-oper_testGetPrefixShardRole: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@4f76bc69 07:09:21,577 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#-1337931814] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#-1337931814] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,577 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Stopping Shard member-1-shard-default-oper_testGetPrefixShardRole 07:09:21,577 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#-1337931814] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#-1337931814] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,577 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 07:09:21,577 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 07:09:21,579 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:09:21,579 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:09:21,582 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 07:09:21,609 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:09:21,617 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-1926092793443747356] 07:09:21,618 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 07:09:21,620 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 07:09:21,620 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 07:09:21,620 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:09:21,623 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:09:21,631 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:21,632 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 07:09:21,632 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 07:09:21,633 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:21,635 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,636 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 07:09:21,640 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#1080205786] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,642 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 07:09:21,643 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 07:09:21,643 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,644 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,644 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:21,644 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,644 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,647 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 07:09:21,649 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@48a27488} 07:09:21,649 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 07:09:21,649 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:21,651 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:21,651 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,651 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,651 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,651 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#1634508006] 07:09:21,651 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,652 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:21,652 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,652 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,652 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-1655379654] 07:09:21,652 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#1634508006] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,652 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-1655379654] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,652 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 07:09:21,653 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Shard created : member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 07:09:21,653 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#690332025 created and ready for shard:member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,653 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-1:snapshot-read#893959159] 07:09:21,653 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#690332025] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,653 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 07:09:21,656 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-1:snapshot-read#2083805205] 07:09:21,656 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 07:09:21,657 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-1:snapshot-read#893959159] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,657 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-1:snapshot-read#2083805205] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,658 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,659 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 07:09:21,659 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#824442787 created and ready for shard:member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,659 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-969023165] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,659 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#824442787] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,660 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:09:21,660 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:09:21,660 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 591 07:09:21,660 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 07:09:21,660 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:21,660 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:09:21,660 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 07:09:21,660 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,661 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,661 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:21,661 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,661 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,661 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 07:09:21,661 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 07:09:21,661 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:09:21,661 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:09:21,661 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 579 07:09:21,661 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 07:09:21,661 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:21,661 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:09:21,662 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 07:09:21,662 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 07:09:21,662 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 07:09:21,662 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:21,662 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 07:09:21,662 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,662 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 07:09:21,662 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:21,663 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 07:09:21,663 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,663 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 07:09:21,663 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 07:09:21,663 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 07:09:21,664 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@2bcadbe3} 07:09:21,664 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 07:09:21,664 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:21,664 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,667 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,667 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,667 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1390830889] 07:09:21,667 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,667 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Shard created : member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 07:09:21,667 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1390830889] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,668 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-1:snapshot-read#1460786412] 07:09:21,668 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,668 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 07:09:21,668 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#2029781628 created and ready for shard:member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,668 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#2029781628] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,669 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,668 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 07:09:21,669 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-861198861] 07:09:21,669 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#369013786 created and ready for shard:member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,669 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-1:snapshot-read#1589611879] 07:09:21,669 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 07:09:21,670 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:09:21,670 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:09:21,670 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 567 07:09:21,670 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 07:09:21,670 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:21,670 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:09:21,670 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:09:21,670 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:09:21,670 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 579 07:09:21,670 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 07:09:21,670 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:21,670 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 07:09:21,670 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:21,670 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:09:21,670 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 07:09:21,670 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 07:09:21,670 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:21,670 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 07:09:21,670 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 07:09:21,670 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 07:09:21,670 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 07:09:21,670 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 07:09:21,671 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,671 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,671 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 07:09:21,671 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 07:09:21,678 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:09:21,689 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-9085127676483620221] 07:09:21,690 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 07:09:21,692 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:09:21,692 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 07:09:21,692 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:09:21,694 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:09:21,704 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:21,710 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:21,715 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,723 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 07:09:21,726 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-292649199] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,731 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 07:09:21,731 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,731 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,733 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 07:09:21,733 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@5d4a00b9} 07:09:21,733 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 07:09:21,734 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:21,734 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:21,734 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,734 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,735 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:21,735 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Shard created : member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 07:09:21,736 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 07:09:21,736 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Shard created : member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 07:09:21,736 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Starting recovery for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 07:09:21,737 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-378273316 created and ready for shard:member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,737 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-2:snapshot-read#812681113] 07:09:21,737 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-1657898118] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,737 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#325481763] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,737 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-378273316] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,737 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1841107392 created and ready for shard:member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,737 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-2:snapshot-read#1252932618] 07:09:21,737 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-2:snapshot-read#812681113] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,737 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,739 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 07:09:21,740 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:09:21,740 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:09:21,740 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 591 07:09:21,740 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 07:09:21,740 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:21,740 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:09:21,743 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 07:09:21,744 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1841107392] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,744 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-2:snapshot-read#1252932618] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,744 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1736966068] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,744 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 07:09:21,744 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 07:09:21,748 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:21,748 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,748 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,748 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,748 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,749 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 07:09:21,751 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:09:21,751 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:09:21,751 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 579 07:09:21,751 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 07:09:21,751 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:21,751 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:09:21,751 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 07:09:21,751 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:21,751 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 07:09:21,751 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,751 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 07:09:21,751 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 07:09:21,751 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:21,751 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 07:09:21,752 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 07:09:21,752 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,752 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,752 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 07:09:21,752 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,752 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,752 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,753 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 07:09:21,753 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 07:09:21,753 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:21,753 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,753 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,775 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 07:09:21,776 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@7256d6dd} 07:09:21,776 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 07:09:21,776 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:21,776 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,777 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,777 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,777 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader on shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-584335945] 07:09:21,777 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,777 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,777 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,777 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader on shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-63125299] 07:09:21,778 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Shard created : member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 07:09:21,778 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-584335945] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,778 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Shard created : member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 07:09:21,778 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 07:09:21,779 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-968772912 created and ready for shard:member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,779 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-2074811794 created and ready for shard:member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,779 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-2:snapshot-read#1267292999] 07:09:21,779 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-2:snapshot-read#-955832193] 07:09:21,779 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-63125299] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,779 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 07:09:21,781 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:09:21,781 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:09:21,781 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 579 07:09:21,781 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 07:09:21,781 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:21,781 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:09:21,782 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 07:09:21,782 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:21,782 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,782 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 07:09:21,782 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 07:09:21,782 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 07:09:21,782 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:09:21,782 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:09:21,782 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 567 07:09:21,782 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,782 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 07:09:21,782 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 07:09:21,782 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:21,782 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:09:21,782 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 07:09:21,782 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 07:09:21,782 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 07:09:21,782 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:21,782 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 07:09:21,782 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,782 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,782 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 07:09:21,788 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:09:21,797 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-4838315378522935747] 07:09:21,797 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 07:09:21,800 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:09:21,800 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 07:09:21,802 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:09:21,802 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:09:21,812 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:21,813 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:21,816 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,820 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 07:09:21,821 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#660702668] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,823 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 07:09:21,823 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,823 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,825 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 07:09:21,826 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@3716c753} 07:09:21,826 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 07:09:21,826 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:21,827 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:21,827 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,828 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:21,830 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,830 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,831 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 07:09:21,831 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-1284934592] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,832 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-3:snapshot-read#-1566925419] 07:09:21,832 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1617793622 created and ready for shard:member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,832 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1617793622] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,832 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-3:snapshot-read#-1566925419] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,832 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 07:09:21,832 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 07:09:21,832 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-301958964] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,833 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#836919597 created and ready for shard:member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,833 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-3:snapshot-read#1818429193] 07:09:21,833 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#836919597] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,833 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 07:09:21,834 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 07:09:21,838 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:09:21,838 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:09:21,838 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 591 07:09:21,838 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 07:09:21,838 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:21,838 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:09:21,838 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 07:09:21,838 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 07:09:21,838 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-3:snapshot-read#1818429193] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,838 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#1331716550] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,839 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:09:21,839 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:09:21,839 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 579 07:09:21,839 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 07:09:21,839 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:21,839 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:09:21,839 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 07:09:21,839 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:21,839 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 07:09:21,839 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 07:09:21,840 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 07:09:21,840 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,843 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 07:09:21,843 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:21,843 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 07:09:21,843 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,844 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 07:09:21,844 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,844 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,845 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 07:09:21,845 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 07:09:21,851 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 07:09:21,852 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@5041b119} 07:09:21,852 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 07:09:21,854 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:21,854 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,854 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,854 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 07:09:21,855 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Shard created : member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 07:09:21,855 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 07:09:21,855 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Shard created : member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 07:09:21,856 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 07:09:21,856 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-334971759] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,856 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#1494983446] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:21,856 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1623142475 created and ready for shard:member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,856 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-3:snapshot-read#-1035916353] 07:09:21,858 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 07:09:21,858 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:21,858 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,858 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,858 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,858 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,858 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:09:21,859 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:09:21,859 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 567 07:09:21,859 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 07:09:21,859 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:21,859 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:09:21,859 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 07:09:21,859 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:21,859 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,859 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:09:21,859 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:09:21,859 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 579 07:09:21,859 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 07:09:21,859 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:21,859 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:09:21,859 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 07:09:21,859 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:21,859 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,859 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 07:09:21,859 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 07:09:21,859 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 07:09:21,859 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 07:09:21,859 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:21,859 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,859 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,859 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,859 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,859 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,859 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,861 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-3:snapshot-read#1758255111] 07:09:21,862 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-226784581 created and ready for shard:member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,862 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 07:09:21,862 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,862 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 07:09:21,862 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 07:09:21,862 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:21,862 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 07:09:22,638 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 07:09:22,638 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 07:09:22,638 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:22,638 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:22,639 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,639 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,639 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,639 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,639 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,639 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,639 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,639 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,639 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:09:22,639 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:09:22,639 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,639 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,639 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,639 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,639 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,639 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,639 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,639 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,639 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,639 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,639 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,639 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,639 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,639 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,639 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,639 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Peer address for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,646 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:22,646 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:22,646 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,646 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,646 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,646 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,646 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,646 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,646 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,646 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,646 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:09:22,646 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,646 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,646 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:09:22,646 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,646 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,646 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,646 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,646 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,646 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,646 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,646 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,646 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,646 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,655 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#1080205786] 07:09:22,655 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:09:22,655 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#1634508006] 07:09:22,655 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#1634508006] 07:09:22,656 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#1080205786] 07:09:22,656 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:09:22,656 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#1634508006] 07:09:22,656 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#1634508006] 07:09:22,656 AM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 07:09:22,656 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,656 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,656 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 07:09:22,656 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 07:09:22,656 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 07:09:22,657 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:09:22,657 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 07:09:22,657 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 07:09:22,657 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 07:09:22,657 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 07:09:22,657 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,657 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]], state: Idle 07:09:22,657 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 07:09:22,657 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Sending local ElectionTimeout to start leader election 07:09:22,657 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 07:09:22,658 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Received TimeoutNow - switching to Candidate 07:09:22,658 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader] 07:09:22,658 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,658 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 07:09:22,658 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]], state: Idle 07:09:22,658 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 07:09:22,658 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Sending local ElectionTimeout to start leader election 07:09:22,658 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote{term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1} to peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,658 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 07:09:22,658 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote{term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1} to peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,658 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Received TimeoutNow - switching to Candidate 07:09:22,658 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 07:09:22,658 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 07:09:22,658 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:22,658 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader] 07:09:22,658 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 07:09:22,658 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 07:09:22,658 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote{term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1} to peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,658 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 07:09:22,658 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote{term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1} to peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,659 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 07:09:22,659 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 07:09:22,659 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:22,659 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 07:09:22,659 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 07:09:22,659 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,659 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]], state: Idle 07:09:22,659 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 07:09:22,659 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Sending local ElectionTimeout to start leader election 07:09:22,659 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 07:09:22,659 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Received TimeoutNow - switching to Candidate 07:09:22,659 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader] 07:09:22,659 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 07:09:22,659 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote{term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1} to peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,659 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote{term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1} to peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,659 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,660 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 07:09:22,660 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 07:09:22,660 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:22,660 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 07:09:22,660 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]], state: Idle 07:09:22,660 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 07:09:22,660 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Sending local ElectionTimeout to start leader election 07:09:22,660 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 07:09:22,660 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 07:09:22,660 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Received TimeoutNow - switching to Candidate 07:09:22,660 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader] 07:09:22,660 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 07:09:22,660 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote{term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1} to peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,660 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote{term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1} to peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,660 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 07:09:22,660 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 07:09:22,660 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:22,661 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:22,661 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 07:09:22,661 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 07:09:22,661 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 07:09:22,661 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1} with higher term appears reachable, updating term. 07:09:22,661 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 07:09:22,661 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote{term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1} - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 07:09:22,661 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 07:09:22,661 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:22,661 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 07:09:22,661 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:22,661 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1} with higher term appears reachable, updating term. 07:09:22,661 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 07:09:22,661 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 07:09:22,661 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1} with higher term appears reachable, updating term. 07:09:22,661 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 07:09:22,661 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote{term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1} - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 07:09:22,661 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote{term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1} - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 07:09:22,661 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 07:09:22,661 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 07:09:22,661 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:22,661 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 07:09:22,661 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1} with higher term appears reachable, updating term. 07:09:22,661 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 07:09:22,661 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote{term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1} - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 07:09:22,661 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 07:09:22,662 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:22,662 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:22,663 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 07:09:22,663 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1} with higher term appears reachable, updating term. 07:09:22,663 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 07:09:22,663 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote{term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1} - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 07:09:22,663 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 07:09:22,663 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:22,663 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 07:09:22,663 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1} with higher term appears reachable, updating term. 07:09:22,663 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:22,663 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 07:09:22,663 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 07:09:22,663 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1} with higher term appears reachable, updating term. 07:09:22,663 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 07:09:22,663 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote{term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1} - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 07:09:22,663 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 07:09:22,663 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote{term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1} - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 07:09:22,663 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 07:09:22,662 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 07:09:22,663 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1} with higher term appears reachable, updating term. 07:09:22,663 AM [cluster-test-shard-dispatcher-36] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 07:09:22,663 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote{term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1} - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 07:09:22,663 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 07:09:22,664 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): RaftRPC message received RequestVoteReply{term=2, voteGranted=true}, my term is 2 07:09:22,664 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): handleRequestVoteReply: RequestVoteReply{term=2, voteGranted=true}, current voteCount: 1 07:09:22,664 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 07:09:22,664 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Election: Leader has following peers: [member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader] 07:09:22,664 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:22,664 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:22,664 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 07:09:22,664 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 07:09:22,664 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 07:09:22,664 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: New leader member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader elected 07:09:22,664 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: New server configuration : [ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]] 07:09:22,664 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 07:09:22,664 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 07:09:22,664 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: ServerOperationContextIdentifier{value=4489fbc9-337c-45c4-8944-0041985989c8}, logIndex: 1, isSendImmediate: true 07:09:22,665 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:22,665 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): RaftRPC message received RequestVoteReply{term=2, voteGranted=true}, my term is 2 07:09:22,665 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:22,665 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): handleRequestVoteReply: RequestVoteReply{term=2, voteGranted=true}, current voteCount: 1 07:09:22,665 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 07:09:22,665 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): RaftRPC message received RequestVoteReply{term=2, voteGranted=true}, my term is 2 07:09:22,665 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): handleRequestVoteReply: RequestVoteReply{term=2, voteGranted=true}, current voteCount: 1 07:09:22,665 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 07:09:22,665 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]] 07:09:22,665 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Election: Leader has following peers: [member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader] 07:09:22,665 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Election: Leader has following peers: [member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader] 07:09:22,665 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:22,665 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:22,665 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:22,665 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: null, logIndex: 2, isSendImmediate: true 07:09:22,665 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:22,665 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:22,665 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,665 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:22,665 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,665 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 07:09:22,665 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 07:09:22,665 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 07:09:22,665 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 07:09:22,665 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 07:09:22,665 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: New leader member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader elected 07:09:22,665 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): RaftRPC message received RequestVoteReply{term=2, voteGranted=true}, my term is 2 07:09:22,665 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): handleRequestVoteReply: RequestVoteReply{term=2, voteGranted=true}, current voteCount: 1 07:09:22,665 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 07:09:22,665 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 07:09:22,665 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: New leader member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader elected 07:09:22,665 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: New server configuration : [ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]] 07:09:22,665 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Election: Leader has following peers: [member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader] 07:09:22,665 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: New server configuration : [ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]] 07:09:22,665 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 07:09:22,665 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 07:09:22,665 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:22,665 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 07:09:22,665 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 07:09:22,665 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:22,665 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 07:09:22,665 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 07:09:22,665 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: ServerOperationContextIdentifier{value=dcbb05b7-ed7c-490d-901b-bcc41a8b7ce3}, logIndex: 1, isSendImmediate: true 07:09:22,665 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 07:09:22,665 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: New leader member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader elected 07:09:22,665 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: ServerOperationContextIdentifier{value=309040f1-ef29-495e-8b85-5fe2c50bc6ce}, logIndex: 1, isSendImmediate: true 07:09:22,665 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: New server configuration : [ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]] 07:09:22,665 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 07:09:22,665 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:22,665 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 07:09:22,665 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:22,665 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:22,666 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:22,666 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: ServerOperationContextIdentifier{value=33c73a48-6674-419f-b43c-ad303aa7f9ba}, logIndex: 1, isSendImmediate: true 07:09:22,666 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]] 07:09:22,666 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:22,666 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]] 07:09:22,666 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:22,666 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:22,666 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:22,666 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:22,666 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]] 07:09:22,666 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:22,666 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: null, logIndex: 2, isSendImmediate: true 07:09:22,666 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: null, logIndex: 2, isSendImmediate: true 07:09:22,666 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:22,666 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:22,666 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,666 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: null, logIndex: 2, isSendImmediate: true 07:09:22,666 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,666 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,666 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,666 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,666 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,666 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 07:09:22,666 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 07:09:22,666 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 07:09:22,666 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 07:09:22,666 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,666 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard cars 07:09:22,666 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,666 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard people 07:09:22,666 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 07:09:22,666 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 07:09:22,666 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 07:09:22,666 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 07:09:22,666 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 07:09:22,666 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 07:09:22,666 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,666 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 07:09:22,666 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard cars 07:09:22,666 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 07:09:22,666 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 07:09:22,666 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,666 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard people 07:09:22,666 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 07:09:22,667 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 07:09:22,667 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 07:09:22,667 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 07:09:22,667 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 07:09:22,667 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 07:09:22,667 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 07:09:22,667 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 07:09:22,667 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 07:09:22,667 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 07:09:22,668 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 07:09:22,668 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 07:09:22,668 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#1080205786] 07:09:22,668 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:09:22,668 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#1634508006] 07:09:22,668 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#1634508006] 07:09:22,669 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 07:09:22,669 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:09:22,669 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 07:09:22,669 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 07:09:22,669 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,669 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,669 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries{term=2, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:22,669 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 07:09:22,669 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 07:09:22,669 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 07:09:22,669 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 07:09:22,669 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 07:09:22,669 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 07:09:22,669 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 07:09:22,669 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 07:09:22,669 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 07:09:22,670 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 07:09:22,670 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#1080205786] 07:09:22,670 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 07:09:22,670 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 07:09:22,670 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-1655379654] 07:09:22,670 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-1655379654] 07:09:22,670 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:22,671 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 07:09:22,671 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 07:09:22,671 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:09:22,671 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 07:09:22,671 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 07:09:22,671 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 07:09:22,671 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,671 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,671 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:09:22,671 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries{term=2, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:22,671 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 07:09:22,671 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,671 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:22,671 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:22,671 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:09:22,671 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 07:09:22,671 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:09:22,671 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,671 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:22,671 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 07:09:22,671 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,671 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:09:22,671 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,671 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,671 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:09:22,671 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,671 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries{term=2, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:22,671 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 07:09:22,671 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:09:22,671 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:09:22,671 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,671 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,671 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,671 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:09:22,671 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,671 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries{term=2, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:22,671 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 07:09:22,671 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:09:22,671 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:22,671 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,671 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:09:22,671 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,671 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-969023165] 07:09:22,671 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 07:09:22,671 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1390830889] 07:09:22,672 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1390830889] 07:09:22,672 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:22,672 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:09:22,672 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:22,672 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:09:22,672 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:22,672 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:09:22,672 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:22,672 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:09:22,672 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-969023165] 07:09:22,672 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 07:09:22,672 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-861198861] 07:09:22,672 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-861198861] 07:09:22,673 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 07:09:22,673 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 07:09:22,673 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 07:09:22,673 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 07:09:22,673 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 07:09:22,673 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 07:09:22,673 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 07:09:22,673 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 07:09:22,673 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 07:09:22,673 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 07:09:22,673 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:22,673 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 07:09:22,673 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 07:09:22,673 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 07:09:22,673 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 07:09:22,673 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-292649199] 07:09:22,673 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 07:09:22,673 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:22,673 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:22,673 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 07:09:22,673 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:09:22,673 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Applying to log - commitIndex: 2, lastAppliedIndex: -1 07:09:22,673 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:22,673 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 0 07:09:22,673 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 07:09:22,673 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 07:09:22,673 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 07:09:22,673 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 07:09:22,673 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:22,673 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 07:09:22,673 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 07:09:22,673 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 1 07:09:22,673 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 07:09:22,673 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 07:09:22,673 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 07:09:22,673 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:22,673 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 2 07:09:22,673 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 07:09:22,673 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 07:09:22,673 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 07:09:22,673 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:22,673 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 07:09:22,673 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 07:09:22,673 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 07:09:22,673 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,673 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:22,673 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:22,673 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 07:09:22,673 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:22,673 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:22,673 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 07:09:22,673 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:22,673 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 07:09:22,673 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] has been successfully replicated to a majority of followers 07:09:22,674 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 07:09:22,674 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 07:09:22,674 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 07:09:22,674 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:22,674 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 07:09:22,674 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 07:09:22,674 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 07:09:22,674 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 07:09:22,674 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 07:09:22,674 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]] - lastApplied: 2, lastIndex: 2 07:09:22,674 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 07:09:22,674 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 07:09:22,674 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 07:09:22,674 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:22,674 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 07:09:22,674 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 07:09:22,673 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:22,674 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 07:09:22,674 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 07:09:22,674 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:22,674 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-1657898118] 07:09:22,674 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Election: Leader has following peers: [member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader] 07:09:22,675 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,675 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,675 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 07:09:22,675 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 07:09:22,675 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 07:09:22,675 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 07:09:22,675 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:22,675 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 07:09:22,675 AM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 07:09:22,675 AM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 07:09:22,674 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-1657898118] 07:09:22,675 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 07:09:22,675 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:09:22,675 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Applying to log - commitIndex: 2, lastAppliedIndex: -1 07:09:22,675 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 0 07:09:22,675 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 07:09:22,675 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 07:09:22,675 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 1 07:09:22,675 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 07:09:22,675 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 07:09:22,675 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 07:09:22,675 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 2 07:09:22,675 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 07:09:22,675 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:22,675 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 07:09:22,675 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 07:09:22,675 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:09:22,675 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,676 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,676 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,676 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 07:09:22,674 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 07:09:22,676 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,676 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,676 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries{term=2, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:22,676 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,676 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] has been successfully replicated to a majority of followers 07:09:22,676 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]] - lastApplied: 2, lastIndex: 2 07:09:22,676 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Election: Leader has following peers: [member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader] 07:09:22,676 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,676 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,676 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,676 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 07:09:22,676 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 07:09:22,676 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 07:09:22,676 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 07:09:22,676 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,676 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 07:09:22,675 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:09:22,676 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Applying to log - commitIndex: 2, lastAppliedIndex: -1 07:09:22,676 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 0 07:09:22,676 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 07:09:22,676 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:22,677 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-292649199] 07:09:22,677 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 07:09:22,677 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 07:09:22,677 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#325481763] 07:09:22,677 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 07:09:22,677 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 07:09:22,677 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 07:09:22,677 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 07:09:22,677 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 07:09:22,677 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 07:09:22,677 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 07:09:22,677 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 07:09:22,677 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 07:09:22,677 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:22,677 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 07:09:22,677 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#325481763] 07:09:22,677 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 07:09:22,677 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,677 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,677 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 07:09:22,677 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 07:09:22,677 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1736966068] 07:09:22,677 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 07:09:22,677 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 07:09:22,677 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-584335945] 07:09:22,677 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Sending AppendEntries to follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries{term=2, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:22,678 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,678 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,678 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,677 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 07:09:22,676 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 07:09:22,679 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 07:09:22,679 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 1 07:09:22,679 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:22,679 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 07:09:22,679 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:22,679 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 07:09:22,679 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 07:09:22,679 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 07:09:22,679 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:22,679 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 2 07:09:22,679 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 07:09:22,679 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 07:09:22,679 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:22,679 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 07:09:22,679 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 07:09:22,679 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 07:09:22,679 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 07:09:22,679 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 07:09:22,679 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,679 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 07:09:22,679 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 07:09:22,679 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 07:09:22,679 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 07:09:22,679 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:22,679 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,677 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries{term=2, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:22,679 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:22,679 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:09:22,679 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 07:09:22,680 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 07:09:22,680 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 07:09:22,680 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,680 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 07:09:22,680 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:09:22,680 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,680 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 07:09:22,680 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 07:09:22,680 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Applying to log - commitIndex: 2, lastAppliedIndex: -1 07:09:22,680 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 0 07:09:22,680 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 07:09:22,680 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 07:09:22,680 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 1 07:09:22,680 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 07:09:22,680 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 07:09:22,680 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 2 07:09:22,680 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 07:09:22,680 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:22,680 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,680 AM [cluster-test-shard-dispatcher-33] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] has been successfully replicated to a majority of followers 07:09:22,680 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]] - lastApplied: 2, lastIndex: 2 07:09:22,680 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Election: Leader has following peers: [member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader] 07:09:22,680 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,680 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,677 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-584335945] 07:09:22,680 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries{term=2, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:22,679 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,680 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 07:09:22,680 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,680 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:22,680 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:22,679 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 07:09:22,680 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 07:09:22,681 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 07:09:22,681 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:22,679 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 07:09:22,681 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 07:09:22,681 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 07:09:22,681 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 07:09:22,681 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 07:09:22,681 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 07:09:22,681 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 07:09:22,682 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:22,682 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:22,682 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 07:09:22,682 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:22,682 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 07:09:22,682 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:22,682 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:22,682 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 07:09:22,682 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] is present: existingEntryTerm: 2 07:09:22,682 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): matchEntry SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] is present: existingEntryTerm: 2 07:09:22,682 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 2, entries to be added from: 2 07:09:22,682 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 07:09:22,682 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 07:09:22,682 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:22,682 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 07:09:22,682 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 07:09:22,682 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 07:09:22,682 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 07:09:22,682 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 07:09:22,682 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 07:09:22,682 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 07:09:22,682 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 07:09:22,682 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 07:09:22,683 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:22,683 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 07:09:22,679 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 07:09:22,681 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 07:09:22,683 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 07:09:22,683 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 07:09:22,683 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,683 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 07:09:22,683 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 07:09:22,683 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 07:09:22,681 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 07:09:22,681 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 07:09:22,681 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 07:09:22,683 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 07:09:22,683 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 07:09:22,683 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 07:09:22,683 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 07:09:22,683 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 07:09:22,683 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 07:09:22,683 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 07:09:22,683 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 07:09:22,683 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 07:09:22,683 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 07:09:22,683 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 07:09:22,683 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:22,683 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 07:09:22,683 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 07:09:22,683 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,680 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries{term=2, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:22,680 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1736966068] 07:09:22,684 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 07:09:22,680 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:22,680 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 07:09:22,684 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 07:09:22,683 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 07:09:22,684 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:22,684 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 07:09:22,684 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:22,684 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 07:09:22,684 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 07:09:22,684 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 07:09:22,684 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 07:09:22,684 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 07:09:22,684 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 07:09:22,684 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 07:09:22,684 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 07:09:22,684 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 07:09:22,684 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 07:09:22,684 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 07:09:22,684 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:22,684 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-63125299] 07:09:22,684 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 07:09:22,684 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:22,684 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 07:09:22,684 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-63125299] 07:09:22,685 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#660702668] 07:09:22,685 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 07:09:22,685 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-1284934592] 07:09:22,685 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 07:09:22,685 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,685 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-1284934592] 07:09:22,685 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#660702668] 07:09:22,685 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 07:09:22,685 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-301958964] 07:09:22,685 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-301958964] 07:09:22,685 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,686 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#1331716550] 07:09:22,686 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,686 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 07:09:22,686 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-334971759] 07:09:22,686 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:22,686 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 07:09:22,686 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 07:09:22,686 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 07:09:22,686 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 07:09:22,686 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 07:09:22,686 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:22,686 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:22,686 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 07:09:22,686 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 07:09:22,686 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:22,686 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 07:09:22,686 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 07:09:22,686 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 07:09:22,686 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 07:09:22,686 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 07:09:22,686 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 07:09:22,686 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 07:09:22,686 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 07:09:22,686 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 07:09:22,686 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:22,686 AM [cluster-test-shard-dispatcher-34] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] has been successfully replicated to a majority of followers 07:09:22,686 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]] - lastApplied: 2, lastIndex: 2 07:09:22,687 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Election: Leader has following peers: [member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader] 07:09:22,687 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,687 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 07:09:22,687 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:22,687 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 07:09:22,687 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-334971759] 07:09:22,687 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#1331716550] 07:09:22,687 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 07:09:22,687 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#1494983446] 07:09:22,688 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,689 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,688 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#1494983446] 07:09:22,689 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 07:09:22,689 AM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 07:09:22,689 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 07:09:22,689 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 07:09:22,689 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 07:09:22,689 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,689 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 07:09:22,689 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 07:09:22,689 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#1080205786] 07:09:22,689 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:09:22,689 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#1634508006] 07:09:22,689 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#1634508006] 07:09:22,690 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-969023165] 07:09:22,690 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 07:09:22,690 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1390830889] 07:09:22,690 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1390830889] 07:09:22,690 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 07:09:22,691 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,691 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,691 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 07:09:22,691 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 07:09:22,691 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 07:09:22,691 AM [cluster-test-shard-dispatcher-36] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,691 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - Stopping Shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,691 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 07:09:22,691 AM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 07:09:22,692 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 07:09:22,692 AM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,692 AM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,692 AM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 07:09:22,692 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 07:09:22,692 AM [cluster-test-shard-dispatcher-36] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,692 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 07:09:22,692 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,692 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 07:09:22,692 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 07:09:22,693 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:09:22,693 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:09:22,696 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:22,696 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:22,697 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 07:09:22,711 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 07:09:22,714 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,714 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,714 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 07:09:22,715 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 07:09:22,715 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 07:09:22,715 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,715 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,715 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 07:09:22,715 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 07:09:22,716 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 07:09:22,716 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,716 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,716 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 07:09:22,716 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 07:09:22,716 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 07:09:22,716 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,716 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,717 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 07:09:22,717 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 07:09:22,717 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 07:09:22,717 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 07:09:22,718 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:09:22,718 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:09:22,721 AM [cluster-test-pekko.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:22,722 AM [cluster-test-pekko.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:22,723 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [01/31/2025 07:09:22.724] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:09:22,728 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 07:09:22,729 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,729 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:22,729 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 07:09:22,729 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 07:09:22,729 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Initiating leader transfer 07:09:22,729 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=null, leaderPayloadVersion=13] 07:09:22,729 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 07:09:22,729 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Initiating leader transfer 07:09:22,729 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=null, leaderPayloadVersion=13] 07:09:22,730 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@58697f69 07:09:22,730 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@58697f69 07:09:22,730 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 07:09:22,730 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Attempting to transfer leadership 07:09:22,730 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,730 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,730 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@42e9c541 07:09:22,730 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@42e9c541 07:09:22,730 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 07:09:22,730 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Attempting to transfer leadership 07:09:22,730 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:22,730 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:23,306 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 07:09:23,306 AM [cluster-test-shard-dispatcher-33] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 576.3 ms 07:09:23,306 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 07:09:23,306 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer failed - sending PoisonPill 07:09:23,306 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Leadership transfer expired 07:09:23,307 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:23,316 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 07:09:23,316 AM [cluster-test-shard-dispatcher-34] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 586.8 ms 07:09:23,316 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 07:09:23,316 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer failed - sending PoisonPill 07:09:23,316 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Leadership transfer expired 07:09:23,316 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 07:09:23,316 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 07:09:23,316 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 07:09:23,317 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 07:09:23,317 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:23,317 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:23,317 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 07:09:23,317 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 07:09:23,317 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Initiating leader transfer 07:09:23,317 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 07:09:23,317 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Initiating leader transfer 07:09:23,317 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4c2e336c 07:09:23,317 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4c2e336c 07:09:23,317 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 07:09:23,317 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Attempting to transfer leadership 07:09:23,317 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@56ae9a14 07:09:23,318 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@56ae9a14 07:09:23,318 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 07:09:23,318 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Attempting to transfer leadership 07:09:23,318 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:23,318 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:23,318 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=null, leaderPayloadVersion=13] 07:09:23,318 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=null, leaderPayloadVersion=13] 07:09:23,318 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:23,318 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:23,636 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 07:09:23,636 AM [cluster-test-shard-dispatcher-33] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 318.8 ms 07:09:23,636 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 07:09:23,636 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer failed - sending PoisonPill 07:09:23,636 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Leadership transfer expired 07:09:23,637 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:23,646 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 07:09:23,646 AM [cluster-test-shard-dispatcher-33] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 329.1 ms 07:09:23,646 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 07:09:23,647 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer failed - sending PoisonPill 07:09:23,647 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Leadership transfer expired 07:09:23,647 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 07:09:23,647 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 07:09:23,647 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 07:09:23,649 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:09:23,649 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:09:23,657 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 07:09:24,669 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [01/31/2025 07:09:24.671] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:09:24,700 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:09:24,706 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-6058438845159217307] 07:09:24,707 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 07:09:24,708 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 07:09:24,711 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 07:09:24,711 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:09:24,711 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:09:24,719 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:24,720 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 07:09:24,720 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 07:09:24,720 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 07:09:24,720 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:24,722 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,725 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 07:09:24,726 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1051394316] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:24,733 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 07:09:24,733 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,733 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,733 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:24,733 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,733 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,736 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 07:09:24,736 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@2db0a2f} 07:09:24,737 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 07:09:24,737 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:24,737 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,737 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:24,737 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,737 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,737 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#1494880591] 07:09:24,737 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,737 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,737 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,737 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#2095856681] 07:09:24,738 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 07:09:24,738 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#1494880591] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:24,738 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#2095856681] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:24,738 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-1:snapshot-read#1771940526] 07:09:24,738 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#2093112692] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:24,738 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-1:snapshot-read#1771940526] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:24,739 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#2093112692 created and ready for shard:member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,739 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 07:09:24,739 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 07:09:24,739 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:24,739 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1874339918 created and ready for shard:member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,739 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-1:snapshot-read#2125036079] 07:09:24,739 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1874339918] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:24,739 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-1:snapshot-read#2125036079] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:24,740 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:09:24,740 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:09:24,740 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 591 07:09:24,740 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 07:09:24,740 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING]] 07:09:24,740 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:24,740 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:09:24,740 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 07:09:24,740 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 07:09:24,740 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:09:24,741 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:09:24,741 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 603 07:09:24,741 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 07:09:24,741 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:24,741 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:09:24,743 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 07:09:24,743 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 07:09:24,743 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:24,743 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 07:09:24,743 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 07:09:24,743 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 07:09:24,743 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:24,743 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,743 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 07:09:24,743 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 07:09:24,743 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,743 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 07:09:24,743 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,744 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-2033867763] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:24,746 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 07:09:24,747 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 07:09:24,747 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,747 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,747 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:24,747 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,747 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,748 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 07:09:24,748 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@249f8a7c} 07:09:24,748 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 07:09:24,749 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:24,749 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,749 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 07:09:24,750 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 07:09:24,751 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,751 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,751 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#1688753182] 07:09:24,751 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,751 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-1:snapshot-read#-320770352] 07:09:24,751 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1992786867 created and ready for shard:member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,752 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 07:09:24,752 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1195653411 created and ready for shard:member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,752 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-1:snapshot-read#-1298070857] 07:09:24,753 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 07:09:24,753 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:09:24,753 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:09:24,753 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 579 07:09:24,753 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING]] 07:09:24,753 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:24,753 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:09:24,753 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 07:09:24,753 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:24,753 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 07:09:24,754 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#1688753182] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:24,754 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1992786867] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:24,754 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,754 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,754 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-348579216] 07:09:24,754 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:09:24,754 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:09:24,754 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 591 07:09:24,755 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 07:09:24,755 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:24,755 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:09:24,755 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 07:09:24,755 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:24,755 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 07:09:24,755 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 07:09:24,755 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 07:09:24,755 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 07:09:24,757 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 07:09:24,757 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,757 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,757 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 07:09:24,757 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 07:09:24,763 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 07:09:24,771 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [1267724803144610763] 07:09:24,772 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 07:09:24,775 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:09:24,775 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 07:09:24,776 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:09:24,776 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:09:24,793 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:24,795 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:24,803 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,812 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1489557938] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:24,813 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 07:09:24,818 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 07:09:24,821 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 07:09:24,821 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,822 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,823 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 07:09:24,823 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 07:09:24,824 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:24,825 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:24,826 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@5edb37a3} 07:09:24,826 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:24,826 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,827 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,827 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Shard created : member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 07:09:24,828 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 07:09:24,828 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Shard created : member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 07:09:24,828 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Starting recovery for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 07:09:24,829 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-96582272] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:24,829 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-576498082] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:24,829 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-2:snapshot-read#2066388560] 07:09:24,830 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,830 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 07:09:24,831 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-2:snapshot-read#2066388560] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:24,831 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1950447044] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:24,831 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 07:09:24,833 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1694880308 created and ready for shard:member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,833 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1694880308] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:24,833 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-2:snapshot-read#-1079499584] 07:09:24,833 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-2:snapshot-read#-1079499584] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:24,833 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1748122578 created and ready for shard:member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,833 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1748122578] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:24,836 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:24,836 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,836 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,836 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,836 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,839 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:09:24,840 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:09:24,840 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 591 07:09:24,840 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 07:09:24,840 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,840 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,840 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:24,840 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,840 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,840 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING]] 07:09:24,840 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:24,841 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:09:24,841 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:09:24,841 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 07:09:24,841 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:24,841 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 07:09:24,841 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 07:09:24,841 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,841 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 07:09:24,841 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,841 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 07:09:24,841 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:09:24,841 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 603 07:09:24,841 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 07:09:24,841 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:24,841 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:09:24,841 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 07:09:24,841 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 07:09:24,841 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 07:09:24,841 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:24,841 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 07:09:24,841 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,841 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,841 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 07:09:24,843 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 07:09:24,843 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@79b86735} 07:09:24,843 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 07:09:24,844 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:24,844 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,844 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Shard created : member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 07:09:24,845 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 07:09:24,845 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,845 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,845 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1127161663] 07:09:24,845 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,845 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,845 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,845 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-2045393219] 07:09:24,846 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Shard created : member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 07:09:24,846 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 07:09:24,847 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-2:snapshot-read#-1873772066] 07:09:24,847 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1810047923 created and ready for shard:member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,847 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1469170019 created and ready for shard:member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,847 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-2:snapshot-read#-1935637419] 07:09:24,847 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1127161663] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:24,847 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-2045393219] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:24,851 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:09:24,852 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:09:24,852 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 591 07:09:24,852 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING]] 07:09:24,852 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:24,852 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:09:24,852 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 07:09:24,852 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 07:09:24,852 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 07:09:24,852 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:24,852 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,851 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:09:24,852 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:09:24,852 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 579 07:09:24,852 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 07:09:24,852 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 07:09:24,852 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,852 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:24,852 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:09:24,852 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 07:09:24,852 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:24,852 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 07:09:24,852 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,853 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 07:09:24,853 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 07:09:24,853 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 07:09:24,853 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,853 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:09:24,853 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 07:09:24,862 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-2970055909594117941] 07:09:24,863 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 07:09:24,865 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:09:24,865 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 07:09:24,867 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:09:24,867 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:09:24,877 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:24,878 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:24,884 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,891 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-1709826600] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:24,894 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 07:09:24,896 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 07:09:24,897 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,897 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,898 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 07:09:24,901 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 07:09:24,902 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:24,902 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@d035497} 07:09:24,902 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:24,902 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,904 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,905 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Shard created : member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 07:09:24,905 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:24,905 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#184314720] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:24,906 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-576492963 created and ready for shard:member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,906 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-3:snapshot-read#-152931268] 07:09:24,906 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-576492963] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:24,906 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-3:snapshot-read#-152931268] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:24,906 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 07:09:24,907 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,908 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Shard created : member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 07:09:24,908 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#275942442] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:24,910 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 07:09:24,912 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1151585251 created and ready for shard:member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,912 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1151585251] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:24,912 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-3:snapshot-read#-1967109505] 07:09:24,912 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-3:snapshot-read#-1967109505] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:24,912 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:09:24,913 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:09:24,913 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 591 07:09:24,913 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING]] 07:09:24,913 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:24,913 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:09:24,913 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 07:09:24,913 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 07:09:24,914 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:09:24,914 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:09:24,914 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 603 07:09:24,914 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING]] 07:09:24,914 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:24,914 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:09:24,915 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 07:09:24,915 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:24,915 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 07:09:24,915 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 07:09:24,915 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,915 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 07:09:24,915 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 07:09:24,917 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#698751424] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:24,917 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 07:09:24,920 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 07:09:24,922 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 07:09:24,922 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 07:09:24,922 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:24,922 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:24,922 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,922 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,922 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,922 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,922 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,922 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,922 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 07:09:24,922 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,922 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 07:09:24,923 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 07:09:24,923 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 07:09:24,924 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 07:09:24,924 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,925 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 07:09:24,925 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,925 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:24,925 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,925 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,925 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@5a0c96e3} 07:09:24,925 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:24,925 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,926 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,926 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,926 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#1875156050] 07:09:24,926 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,926 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,926 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,926 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-839765363] 07:09:24,926 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Shard created : member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 07:09:24,926 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Shard created : member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 07:09:24,926 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#1875156050] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:24,926 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-839765363] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:24,926 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1130394416 created and ready for shard:member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,926 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-773307282 created and ready for shard:member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,926 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-3:snapshot-read#1845757159] 07:09:24,927 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 07:09:24,927 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-3:snapshot-read#1984853027] 07:09:24,927 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 07:09:24,928 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:09:24,928 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:09:24,928 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 579 07:09:24,928 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 07:09:24,928 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:24,928 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:09:24,928 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 07:09:24,928 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:24,928 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 07:09:24,928 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,928 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 07:09:24,928 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 07:09:24,928 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 07:09:24,928 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 07:09:24,928 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,928 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 591 07:09:24,928 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 07:09:24,928 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 07:09:24,928 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:24,928 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 07:09:24,928 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 07:09:24,928 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 07:09:24,929 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 07:09:24,929 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:24,929 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 07:09:24,929 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,929 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:24,929 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 07:09:25,75 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 07:09:25,75 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Received ElectionTimeout - switching to Candidate 07:09:25,76 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown] 07:09:25,76 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 07:09:25,76 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 07:09:25,76 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 07:09:25,76 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:25,76 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 07:09:25,76 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 07:09:25,125 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 07:09:25,125 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Received ElectionTimeout - switching to Candidate 07:09:25,125 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown] 07:09:25,125 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 07:09:25,125 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 07:09:25,125 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 07:09:25,125 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:25,125 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 07:09:25,125 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Received ElectionTimeout - switching to Candidate 07:09:25,125 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown] 07:09:25,125 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 07:09:25,125 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 07:09:25,125 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 07:09:25,125 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:25,125 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 07:09:25,125 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 07:09:25,126 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 07:09:25,126 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 07:09:25,165 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 07:09:25,165 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Received ElectionTimeout - switching to Candidate 07:09:25,165 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Election: Candidate has following voting peers: [member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown] 07:09:25,165 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 07:09:25,165 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 07:09:25,165 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 07:09:25,165 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:25,165 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 07:09:25,165 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 07:09:25,415 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 07:09:25,415 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 07:09:25,485 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 07:09:25,485 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 07:09:25,486 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 07:09:25,486 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 07:09:25,494 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 07:09:25,495 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 07:09:25,726 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 07:09:25,726 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 07:09:25,727 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:25,727 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,727 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:25,727 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,727 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,727 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,727 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,727 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,727 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,727 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:09:25,727 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,727 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,727 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,727 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,727 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,727 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,727 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Peer address for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,727 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,727 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,727 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,727 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:09:25,727 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,727 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,727 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,727 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,727 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,727 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,728 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,734 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:25,734 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,734 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,734 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,734 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,734 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:09:25,734 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,734 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:25,734 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,734 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,734 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,734 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,734 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,734 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,734 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,734 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,734 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,734 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,734 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:09:25,734 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,734 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,734 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,734 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,734 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,785 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 07:09:25,785 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 07:09:25,785 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote{term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1} to peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,786 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote{term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1} to peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,787 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:25,787 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 07:09:25,788 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote{term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1} with higher term appears reachable, updating term. 07:09:25,788 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 07:09:25,788 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote{term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1} - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 07:09:25,788 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply{term=4, voteGranted=true} 07:09:25,788 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:25,788 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 07:09:25,788 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote{term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1} with higher term appears reachable, updating term. 07:09:25,788 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 07:09:25,788 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote{term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1} - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 07:09:25,788 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply{term=4, voteGranted=true} 07:09:25,790 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): RaftRPC message received RequestVoteReply{term=4, voteGranted=true}, my term is 4 07:09:25,790 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): handleRequestVoteReply: RequestVoteReply{term=4, voteGranted=true}, current voteCount: 1 07:09:25,790 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 07:09:25,790 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Election: Leader has following peers: [member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown] 07:09:25,790 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:25,790 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:25,790 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 07:09:25,790 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 07:09:25,790 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 07:09:25,790 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 07:09:25,790 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:25,791 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:25,791 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 07:09:25,791 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 07:09:25,791 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 07:09:25,791 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:25,791 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:25,792 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:25,792 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 07:09:25,792 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 07:09:25,792 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 07:09:25,793 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 07:09:25,793 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 07:09:25,793 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 07:09:25,793 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 07:09:25,793 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:25,793 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,793 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 07:09:25,793 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries{term=4, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:25,795 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 07:09:25,795 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,796 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries{term=4, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:25,796 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 07:09:25,796 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 07:09:25,796 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:25,796 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:25,796 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 07:09:25,796 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply{term=4, success=true, followerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:25,797 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:25,797 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 07:09:25,797 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 07:09:25,797 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:25,797 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,797 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries{term=4, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:25,798 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 07:09:25,798 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,798 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries{term=4, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:25,798 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 07:09:25,798 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 07:09:25,798 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:25,798 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:25,799 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 07:09:25,799 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply{term=4, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:25,799 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 07:09:25,799 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 07:09:25,799 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 07:09:25,799 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 0 07:09:25,799 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 07:09:25,799 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 07:09:25,799 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 1 07:09:25,799 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 07:09:25,799 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:25,799 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:25,799 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:25,799 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]] - lastApplied: 1, lastIndex: 1 07:09:25,800 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Election: Leader has following peers: [member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown] 07:09:25,800 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:25,800 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,800 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries{term=4, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:25,800 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:25,800 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 07:09:25,800 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 07:09:25,800 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 07:09:25,800 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 07:09:25,800 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:25,800 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 07:09:25,800 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 07:09:25,801 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 07:09:25,801 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 07:09:25,801 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 07:09:25,801 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 07:09:25,801 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 07:09:25,801 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 07:09:25,801 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 07:09:25,801 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:25,801 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 07:09:25,802 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries{term=4, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:25,802 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 07:09:25,802 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] is present: existingEntryTerm: 4 07:09:25,802 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 1 07:09:25,802 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 07:09:25,802 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 07:09:25,802 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply{term=4, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:25,802 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 07:09:25,802 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 07:09:25,802 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 07:09:25,802 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 07:09:25,802 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 07:09:25,802 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 07:09:25,802 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:25,802 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 07:09:25,808 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:25,808 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:25,808 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,808 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,808 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,808 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,808 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,808 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,808 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,808 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,808 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:09:25,808 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,808 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:09:25,808 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,808 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,808 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,808 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,808 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,808 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,808 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,808 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,808 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,808 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,808 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,865 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 07:09:25,865 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 07:09:25,865 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote{term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1} to peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,865 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote{term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1} to peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,867 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:25,867 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:25,867 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 07:09:25,867 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote{term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1} with higher term appears reachable, updating term. 07:09:25,867 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 07:09:25,867 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 07:09:25,867 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote{term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1} with higher term appears reachable, updating term. 07:09:25,867 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 07:09:25,867 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote{term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1} - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 07:09:25,867 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote{term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1} - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 07:09:25,867 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply{term=4, voteGranted=true} 07:09:25,867 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply{term=4, voteGranted=true} 07:09:25,869 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): RaftRPC message received RequestVoteReply{term=4, voteGranted=true}, my term is 4 07:09:25,869 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): handleRequestVoteReply: RequestVoteReply{term=4, voteGranted=true}, current voteCount: 1 07:09:25,869 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 07:09:25,869 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Election: Leader has following peers: [member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown] 07:09:25,869 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:25,870 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:25,870 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 07:09:25,870 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 07:09:25,870 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 07:09:25,870 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:25,870 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 07:09:25,870 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:25,870 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 07:09:25,870 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 07:09:25,870 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:25,870 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 07:09:25,870 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:25,871 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:25,871 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 07:09:25,871 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 07:09:25,872 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 07:09:25,872 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 07:09:25,872 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 07:09:25,872 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 07:09:25,872 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 07:09:25,873 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 07:09:25,873 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:25,873 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,873 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries{term=4, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:25,873 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 07:09:25,873 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 07:09:25,873 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:25,873 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,873 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries{term=4, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:25,873 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 07:09:25,873 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,873 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 07:09:25,873 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,874 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries{term=4, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:25,874 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries{term=4, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:25,874 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 07:09:25,874 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 07:09:25,874 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 07:09:25,874 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 07:09:25,874 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:25,874 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:25,874 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:25,874 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:25,874 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 07:09:25,874 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply{term=4, success=true, followerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:25,874 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 07:09:25,874 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply{term=4, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:25,875 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:25,875 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:25,875 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 07:09:25,875 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 07:09:25,875 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 07:09:25,875 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 07:09:25,876 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote{term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1} to peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,876 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote{term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1} to peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,876 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote{term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1} to peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,876 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote{term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1} to peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,876 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 07:09:25,876 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 07:09:25,876 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 07:09:25,876 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 0 07:09:25,876 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 07:09:25,876 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 07:09:25,876 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 1 07:09:25,876 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 07:09:25,876 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:25,876 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:25,876 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 07:09:25,876 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:25,876 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]] - lastApplied: 1, lastIndex: 1 07:09:25,876 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Election: Leader has following peers: [member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown] 07:09:25,876 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:25,876 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:25,876 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 07:09:25,876 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 07:09:25,876 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 07:09:25,876 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 07:09:25,876 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 07:09:25,879 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:25,879 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 07:09:25,879 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote{term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1} with higher term appears reachable, updating term. 07:09:25,879 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 07:09:25,879 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote{term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1} - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 07:09:25,879 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply{term=4, voteGranted=true} 07:09:25,879 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:25,879 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:25,879 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 07:09:25,879 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote{term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1} with higher term appears reachable, updating term. 07:09:25,879 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 07:09:25,879 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 07:09:25,879 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote{term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1} - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 07:09:25,879 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote{term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1} with higher term appears reachable, updating term. 07:09:25,879 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply{term=4, voteGranted=true} 07:09:25,879 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 07:09:25,879 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote{term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1} - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 07:09:25,879 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 07:09:25,879 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 07:09:25,879 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply{term=4, voteGranted=true} 07:09:25,879 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 07:09:25,879 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 07:09:25,879 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 07:09:25,879 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 07:09:25,879 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 07:09:25,879 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:25,879 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:25,879 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 07:09:25,879 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote{term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1} with higher term appears reachable, updating term. 07:09:25,879 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 07:09:25,879 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 07:09:25,879 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote{term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1} - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 07:09:25,879 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply{term=4, voteGranted=true} 07:09:25,879 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 07:09:25,879 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 07:09:25,879 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 07:09:25,879 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 07:09:25,879 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 07:09:25,879 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 07:09:25,879 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 07:09:25,879 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:25,880 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 07:09:25,880 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): RaftRPC message received RequestVoteReply{term=4, voteGranted=true}, my term is 4 07:09:25,880 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): RaftRPC message received RequestVoteReply{term=4, voteGranted=true}, my term is 4 07:09:25,880 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): handleRequestVoteReply: RequestVoteReply{term=4, voteGranted=true}, current voteCount: 1 07:09:25,880 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): handleRequestVoteReply: RequestVoteReply{term=4, voteGranted=true}, current voteCount: 1 07:09:25,880 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 07:09:25,880 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 07:09:25,880 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Election: Leader has following peers: [member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown] 07:09:25,880 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:25,880 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Election: Leader has following peers: [member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown] 07:09:25,880 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:25,880 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 07:09:25,880 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:25,880 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 07:09:25,881 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 07:09:25,881 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 07:09:25,881 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:25,881 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 07:09:25,881 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:25,881 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 07:09:25,881 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:25,881 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 07:09:25,881 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 07:09:25,881 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 07:09:25,881 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:25,881 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:25,881 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:25,881 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:25,881 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 07:09:25,881 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:25,881 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 07:09:25,881 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:25,881 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 07:09:25,881 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 07:09:25,881 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 07:09:25,881 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:25,881 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:25,882 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 07:09:25,882 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 07:09:25,882 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 07:09:25,882 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 07:09:25,882 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStatesWithVotingMembersDown is ready 07:09:25,883 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 07:09:25,883 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStatesWithVotingMembersDown is ready 07:09:25,883 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 07:09:25,883 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 07:09:25,883 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStatesWithVotingMembersDown is ready 07:09:25,883 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 07:09:25,883 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 07:09:25,883 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 07:09:25,883 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 07:09:25,883 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 07:09:25,883 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:25,883 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,883 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 07:09:25,883 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries{term=4, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:25,884 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 07:09:25,884 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,884 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 07:09:25,884 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 07:09:25,884 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:25,884 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,884 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries{term=4, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:25,884 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 07:09:25,884 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,884 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 07:09:25,884 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 07:09:25,884 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:25,884 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,884 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries{term=4, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:25,884 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 07:09:25,884 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 07:09:25,884 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:25,884 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,884 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries{term=4, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:25,884 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 07:09:25,884 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,884 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 07:09:25,884 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,884 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries{term=4, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:25,884 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 07:09:25,884 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 07:09:25,884 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:25,884 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:25,884 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries{term=4, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:25,885 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries{term=4, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:25,885 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 07:09:25,885 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 07:09:25,885 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 07:09:25,885 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply{term=4, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:25,885 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:25,885 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:25,885 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 07:09:25,885 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply{term=4, success=true, followerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:25,885 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:25,885 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 07:09:25,885 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 07:09:25,885 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:25,885 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 07:09:25,885 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 0 07:09:25,885 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 07:09:25,885 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 07:09:25,885 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 1 07:09:25,885 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 07:09:25,885 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:25,885 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:25,885 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]] - lastApplied: 1, lastIndex: 1 07:09:25,885 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown] 07:09:25,886 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:25,886 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,886 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries{term=4, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:25,886 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:25,886 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 07:09:25,886 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 07:09:25,886 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 07:09:25,886 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 07:09:25,886 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 07:09:25,886 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:25,886 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 07:09:25,886 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStatesWithVotingMembersDown is ready 07:09:25,886 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} is now ready 07:09:25,886 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 07:09:25,886 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries{term=4, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:25,886 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 07:09:25,886 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 07:09:25,886 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] is present: existingEntryTerm: 4 07:09:25,886 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 1 07:09:25,886 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 07:09:25,886 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 07:09:25,886 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 07:09:25,886 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply{term=4, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:25,886 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 07:09:25,886 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 07:09:25,887 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 07:09:25,887 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 07:09:25,887 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 07:09:25,887 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 07:09:25,887 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 07:09:25,887 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 07:09:25,887 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 07:09:25,887 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 07:09:25,887 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 07:09:25,887 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:25,887 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:25,887 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 07:09:25,887 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 07:09:25,887 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 07:09:25,887 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 07:09:25,887 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:25,887 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:25,887 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 07:09:25,887 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply{term=4, success=true, followerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:25,888 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 07:09:25,888 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:25,888 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 07:09:25,888 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 07:09:25,888 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 07:09:25,888 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 0 07:09:25,888 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 07:09:25,888 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 07:09:25,888 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 1 07:09:25,888 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 07:09:25,888 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:25,888 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:25,888 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]] - lastApplied: 1, lastIndex: 1 07:09:25,888 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Election: Leader has following peers: [member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown] 07:09:25,888 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:25,888 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,888 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries{term=4, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:25,888 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 07:09:25,889 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 07:09:25,889 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 07:09:25,889 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 07:09:25,889 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 07:09:25,889 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 07:09:25,889 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStatesWithVotingMembersDown is ready 07:09:25,889 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} is now ready 07:09:25,889 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1051394316] 07:09:25,889 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:09:25,889 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#1494880591] 07:09:25,889 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#1494880591] 07:09:25,889 AM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 07:09:25,889 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,889 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 07:09:25,890 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,890 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 07:09:25,890 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 07:09:25,890 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 07:09:25,890 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 07:09:25,890 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 07:09:25,890 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 07:09:25,890 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 07:09:25,890 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:25,890 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 07:09:25,890 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 07:09:25,890 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:09:25,890 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 07:09:25,890 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 07:09:25,890 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 07:09:25,890 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 07:09:25,890 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 07:09:25,891 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,891 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,891 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,891 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,892 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]], state: Idle 07:09:25,892 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true}, serversVisited=[]], state: Idle 07:09:25,892 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 07:09:25,892 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 07:09:25,892 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 07:09:25,892 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 07:09:25,892 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: New server configuration : [ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]] 07:09:25,892 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]] 07:09:25,892 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 07:09:25,892 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 07:09:25,892 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 07:09:25,892 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 07:09:25,892 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]], state: Idle 07:09:25,892 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=ac20e949-a85c-4e30-b7a2-598a5531755b}, logIndex: 2, isSendImmediate: true 07:09:25,892 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 07:09:25,892 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=279ba535-2890-4fcc-81a5-af0b5ec76ae5}, logIndex: 2, isSendImmediate: true 07:09:25,892 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:25,892 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,892 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING]] 07:09:25,892 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:25,892 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,892 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: New server configuration : [ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]] 07:09:25,892 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:25,892 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,892 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 07:09:25,892 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries{term=4, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]} 07:09:25,892 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 07:09:25,892 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries{term=4, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]} 07:09:25,892 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 07:09:25,892 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=3008bce5-5c98-4128-9262-4e316f58b6d7}, logIndex: 2, isSendImmediate: true 07:09:25,892 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:25,892 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,892 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,892 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]], state: Idle 07:09:25,892 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:25,892 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries{term=4, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]} 07:09:25,892 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,892 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 07:09:25,892 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries{term=4, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]} 07:09:25,892 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING]] 07:09:25,892 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: New server configuration : [ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]] 07:09:25,892 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 07:09:25,892 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:25,892 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,892 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 07:09:25,892 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true}, serversVisited=[]] 07:09:25,892 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries{term=4, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]} 07:09:25,892 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard cars 07:09:25,892 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=1e70bf1d-ab9b-455f-97ab-6e4327df6d28}, logIndex: 2, isSendImmediate: true 07:09:25,892 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 07:09:25,892 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 07:09:25,893 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:25,893 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 07:09:25,893 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,893 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,893 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard cars 07:09:25,893 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,893 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard people 07:09:25,893 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 07:09:25,893 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries{term=4, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]} 07:09:25,893 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 07:09:25,893 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 07:09:25,893 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 07:09:25,893 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:25,893 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 07:09:25,893 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,893 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries{term=4, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]} 07:09:25,893 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 07:09:25,893 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,893 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 07:09:25,893 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard people 07:09:25,893 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 07:09:25,893 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 07:09:25,893 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 07:09:25,893 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1051394316] 07:09:25,893 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 07:09:25,893 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:09:25,893 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#1494880591] 07:09:25,893 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#1494880591] 07:09:25,894 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1051394316] 07:09:25,894 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 07:09:25,894 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,894 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,894 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries{term=4, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]} 07:09:25,894 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 07:09:25,894 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 07:09:25,894 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 07:09:25,894 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 07:09:25,894 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 07:09:25,894 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 07:09:25,894 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries{term=4, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]} 07:09:25,895 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries{term=4, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]} 07:09:25,895 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 07:09:25,895 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 07:09:25,895 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 07:09:25,895 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 07:09:25,895 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 07:09:25,895 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 07:09:25,895 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 07:09:25,895 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 07:09:25,895 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#2095856681] 07:09:25,895 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#2095856681] 07:09:25,895 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 07:09:25,895 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 07:09:25,895 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply{term=4, success=true, followerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:25,895 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries{term=4, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]} 07:09:25,895 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 07:09:25,895 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 07:09:25,895 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 07:09:25,895 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 07:09:25,895 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply{term=4, success=true, followerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:25,895 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING]] 07:09:25,895 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 07:09:25,895 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply{term=4, success=true, followerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:25,895 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 07:09:25,895 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 07:09:25,895 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 07:09:25,896 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-2033867763] 07:09:25,896 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 07:09:25,896 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#1688753182] 07:09:25,896 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#1688753182] 07:09:25,896 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries{term=4, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]} 07:09:25,896 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries{term=4, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]} 07:09:25,896 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 07:09:25,896 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 07:09:25,896 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 07:09:25,896 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 07:09:25,896 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 07:09:25,896 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 07:09:25,896 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 07:09:25,896 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 07:09:25,896 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 07:09:25,896 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING]] 07:09:25,896 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:25,896 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 07:09:25,896 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply{term=4, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:25,896 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 07:09:25,896 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:25,896 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:25,896 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries{term=4, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]} 07:09:25,896 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 07:09:25,896 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 07:09:25,896 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 07:09:25,896 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 07:09:25,896 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 07:09:25,896 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 07:09:25,896 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply{term=4, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:25,896 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-2033867763] 07:09:25,897 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 07:09:25,897 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 07:09:25,897 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-348579216] 07:09:25,897 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 07:09:25,897 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-348579216] 07:09:25,897 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 07:09:25,897 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:25,897 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:25,897 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING]] 07:09:25,897 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 07:09:25,897 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply{term=4, success=true, followerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:25,897 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1489557938] 07:09:25,897 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries{term=4, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:25,897 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 07:09:25,897 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 07:09:25,897 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:25,897 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:25,897 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 07:09:25,897 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 07:09:25,897 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-96582272] 07:09:25,897 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply{term=4, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:25,897 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 07:09:25,897 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] 07:09:25,898 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries{term=4, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85]]} 07:09:25,898 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 07:09:25,897 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-96582272] 07:09:25,898 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 07:09:25,898 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85] is present: existingEntryTerm: 4 07:09:25,898 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:25,898 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 1 07:09:25,898 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,898 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 07:09:25,898 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 07:09:25,898 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries{term=4, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]} 07:09:25,898 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply{term=4, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:25,898 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 07:09:25,898 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 07:09:25,898 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 07:09:25,898 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 07:09:25,898 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 07:09:25,898 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 07:09:25,898 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 07:09:25,898 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@72c93d85 07:09:25,898 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:25,898 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 07:09:25,899 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 07:09:25,899 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStatesWithVotingMembersDown is ready 07:09:25,899 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,899 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries{term=4, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]} 07:09:25,899 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 07:09:25,899 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 07:09:25,899 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 07:09:25,899 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 07:09:25,899 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING]] 07:09:25,899 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 07:09:25,899 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply{term=4, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:25,899 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 07:09:25,898 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 07:09:25,900 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 07:09:25,900 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:25,899 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 07:09:25,900 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 07:09:25,900 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply{term=4, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:25,900 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 07:09:25,900 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:25,900 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 07:09:25,900 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1489557938] 07:09:25,900 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 07:09:25,900 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-576498082] 07:09:25,900 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-576498082] 07:09:25,901 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 07:09:25,901 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1950447044] 07:09:25,901 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 07:09:25,901 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1127161663] 07:09:25,901 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1127161663] 07:09:25,901 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1950447044] 07:09:25,901 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 07:09:25,902 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-2045393219] 07:09:25,902 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-2045393219] 07:09:25,902 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-1709826600] 07:09:25,902 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 07:09:25,902 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#184314720] 07:09:25,902 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#184314720] 07:09:25,902 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-1709826600] 07:09:25,902 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 07:09:25,902 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#275942442] 07:09:25,902 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#275942442] 07:09:25,902 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#698751424] 07:09:25,902 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 07:09:25,902 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#1875156050] 07:09:25,902 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#1875156050] 07:09:25,902 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#698751424] 07:09:25,902 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 07:09:25,902 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-839765363] 07:09:25,902 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-839765363] 07:09:25,903 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1051394316] 07:09:25,903 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:09:25,903 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#1494880591] 07:09:25,903 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#1494880591] 07:09:25,903 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 07:09:25,903 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,903 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 07:09:25,904 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,904 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,904 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 07:09:25,904 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,904 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 07:09:25,904 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 07:09:25,904 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 07:09:25,904 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 07:09:25,905 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,905 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 07:09:25,905 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,905 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,905 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 07:09:25,905 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,905 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 07:09:25,905 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 07:09:25,905 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 07:09:25,906 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:09:25,906 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:09:25,909 AM [cluster-test-pekko.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:25,909 AM [cluster-test-pekko.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:25,909 AM [cluster-test-pekko.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:25,909 AM [cluster-test-pekko.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:25,928 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 07:09:25,928 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,929 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,929 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 07:09:25,929 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 07:09:25,929 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 07:09:25,929 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,929 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - Stopping Shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,929 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 07:09:25,930 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 07:09:25,930 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 07:09:25,931 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,931 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,931 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 07:09:25,931 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 07:09:25,931 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 07:09:25,931 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,931 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:25,935 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 07:09:25,935 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 07:09:25,936 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:09:25,936 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:09:26,506 AM [cluster-test-shard-dispatcher-31] [WARN] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 07:09:26,506 AM [cluster-test-shard-dispatcher-13] [WARN] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true}, serversVisited=[]] 07:09:26,506 AM [cluster-test-shard-dispatcher-32] [WARN] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 07:09:26,506 AM [cluster-test-shard-dispatcher-31] [WARN] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 07:09:26,746 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 07:09:26,925 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-2970055909594117941] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 07:09:26,955 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [1267724803144610763] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 07:09:26,956 AM [cluster-test-pekko.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:26,957 AM [cluster-test-pekko.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:26,958 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [01/31/2025 07:09:26.960] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:09:26,976 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 07:09:26,976 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:26,977 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:26,977 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 07:09:26,977 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 07:09:26,977 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Initiating leader transfer 07:09:26,977 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 07:09:26,977 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Initiating leader transfer 07:09:26,977 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7b89ca73 07:09:26,977 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@276a0a93 07:09:26,977 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7b89ca73 07:09:26,977 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@276a0a93 07:09:26,977 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 07:09:26,977 AM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 07:09:26,977 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Attempting to transfer leadership 07:09:26,977 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Attempting to transfer leadership 07:09:26,977 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:26,977 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:26,977 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:26,977 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:26,977 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=null, leaderPayloadVersion=13] 07:09:26,978 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=null, leaderPayloadVersion=13] 07:09:27,395 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 07:09:27,395 AM [cluster-test-shard-dispatcher-18] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 417.8 ms 07:09:27,395 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 07:09:27,395 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer failed - sending PoisonPill 07:09:27,395 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Leadership transfer expired 07:09:27,396 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Stopping Shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:27,485 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 07:09:27,485 AM [cluster-test-shard-dispatcher-31] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 508.6 ms 07:09:27,485 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 07:09:27,485 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer failed - sending PoisonPill 07:09:27,485 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Leadership transfer expired 07:09:27,486 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 07:09:27,486 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 07:09:27,486 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 07:09:27,487 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 07:09:27,487 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:27,487 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:27,487 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 07:09:27,488 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 07:09:27,488 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Initiating leader transfer 07:09:27,488 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 07:09:27,488 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Initiating leader transfer 07:09:27,488 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=null, leaderPayloadVersion=13] 07:09:27,488 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=null, leaderPayloadVersion=13] 07:09:27,488 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7c268e2b 07:09:27,488 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7c268e2b 07:09:27,488 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 07:09:27,488 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Attempting to transfer leadership 07:09:27,488 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:27,488 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@29adf00e 07:09:27,488 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:27,488 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@29adf00e 07:09:27,488 AM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 07:09:27,488 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Attempting to transfer leadership 07:09:27,488 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:27,488 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 07:09:27,795 AM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 07:09:27,795 AM [cluster-test-shard-dispatcher-32] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 307.2 ms 07:09:27,795 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 07:09:27,795 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer failed - sending PoisonPill 07:09:27,795 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Leadership transfer expired 07:09:27,796 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:27,805 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 07:09:27,805 AM [cluster-test-shard-dispatcher-18] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 317.3 ms 07:09:27,805 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 07:09:27,805 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer failed - sending PoisonPill 07:09:27,805 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Leadership transfer expired 07:09:27,806 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 07:09:27,806 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 07:09:27,806 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 07:09:27,807 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:09:27,808 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:09:27,810 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [01/31/2025 07:09:27.812] [cluster-test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:09:27,830 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:09:27,842 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [2680890936978896764] 07:09:27,842 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 07:09:27,845 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 07:09:27,845 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 07:09:27,845 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:09:27,845 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:09:27,858 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:27,859 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:27,861 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardLeaderReplica 07:09:27,862 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 07:09:27,862 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 07:09:27,862 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 07:09:27,865 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-1698317565] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:27,866 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 07:09:27,868 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 07:09:27,868 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testRemoveShardLeaderReplica 07:09:27,868 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testRemoveShardLeaderReplica 07:09:27,868 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:27,868 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 07:09:27,868 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 07:09:27,871 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 07:09:27,873 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@124efd4a} 07:09:27,873 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} started 07:09:27,873 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:27,873 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testRemoveShardLeaderReplica 07:09:27,873 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:27,874 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 07:09:27,874 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 07:09:27,874 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica on shard member-1-shard-cars-config_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-372822596] 07:09:27,874 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testRemoveShardLeaderReplica 07:09:27,874 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 07:09:27,874 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 07:09:27,874 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica on shard member-1-shard-people-config_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica#858071892] 07:09:27,874 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-config_testRemoveShardLeaderReplica, persistent : true 07:09:27,874 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-372822596] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:27,875 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica-notifier#-1106818465 created and ready for shard:member-1-shard-cars-config_testRemoveShardLeaderReplica 07:09:27,875 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica/shard-member-1:snapshot-read#684951935] 07:09:27,875 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica-notifier#-1106818465] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:27,875 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Shard created : member-1-shard-people-config_testRemoveShardLeaderReplica, persistent : true 07:09:27,875 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testRemoveShardLeaderReplica with journal batch size 1 07:09:27,875 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica/shard-member-1:snapshot-read#684951935] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:27,875 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica#858071892] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:27,875 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:27,875 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica-notifier#-1137574616 created and ready for shard:member-1-shard-people-config_testRemoveShardLeaderReplica 07:09:27,875 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica-notifier#-1137574616] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:27,875 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:27,876 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica/shard-member-1:snapshot-read#503356590] 07:09:27,876 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:27,876 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica/shard-member-1:snapshot-read#503356590] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:27,876 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-1-shard-people-config_testRemoveShardLeaderReplica with journal batch size 1 07:09:27,876 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 07:09:27,876 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testRemoveShardLeaderReplica is initialized 07:09:27,876 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:09:27,876 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:27,876 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from null to Follower 07:09:27,876 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 07:09:27,876 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from null to Follower 07:09:27,877 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardLeaderReplica 07:09:27,878 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#-1435338125] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:27,878 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 07:09:27,879 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:27,879 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:27,879 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 07:09:27,879 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testRemoveShardLeaderReplica 07:09:27,879 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:09:27,879 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:27,879 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 07:09:27,879 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testRemoveShardLeaderReplica is initialized 07:09:27,879 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from null to Follower 07:09:27,880 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testRemoveShardLeaderReplica 07:09:27,880 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 07:09:27,880 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:27,880 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 07:09:27,880 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 07:09:27,882 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 07:09:27,882 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from null to Follower 07:09:27,882 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@26e17867} 07:09:27,882 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} started 07:09:27,882 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:27,882 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testRemoveShardLeaderReplica 07:09:27,886 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 07:09:27,886 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 07:09:27,886 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica on shard member-1-shard-cars-oper_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica#1408839362] 07:09:27,886 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testRemoveShardLeaderReplica 07:09:27,886 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-oper_testRemoveShardLeaderReplica, persistent : true 07:09:27,887 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica#1408839362] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:27,887 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 07:09:27,887 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 07:09:27,887 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica on shard member-1-shard-people-oper_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica#1587643900] 07:09:27,887 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica-notifier#-581904329 created and ready for shard:member-1-shard-cars-oper_testRemoveShardLeaderReplica 07:09:27,887 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica/shard-member-1:snapshot-read#2129227941] 07:09:27,887 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testRemoveShardLeaderReplica with journal batch size 1 07:09:27,887 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica-notifier#-581904329] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:27,888 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:27,888 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-people-oper_testRemoveShardLeaderReplica, persistent : true 07:09:27,888 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:27,888 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:09:27,888 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:27,888 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from null to Follower 07:09:27,888 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 07:09:27,888 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testRemoveShardLeaderReplica is initialized 07:09:27,888 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 07:09:27,888 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from null to Follower 07:09:27,888 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica-notifier#-1213001316 created and ready for shard:member-1-shard-people-oper_testRemoveShardLeaderReplica 07:09:27,888 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica/shard-member-1:snapshot-read#-823439431] 07:09:27,888 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testRemoveShardLeaderReplica with journal batch size 1 07:09:27,889 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:27,889 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:27,889 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 07:09:27,889 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testRemoveShardLeaderReplica is initialized 07:09:27,889 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:09:27,889 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:27,889 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from null to Follower 07:09:27,889 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 07:09:27,890 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from null to Follower 07:09:27,891 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:09:27,899 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-7040090376830374142] 07:09:27,899 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 07:09:27,902 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:09:27,902 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 07:09:27,903 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:09:27,903 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:09:27,913 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:27,916 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:27,919 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardLeaderReplica 07:09:27,923 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-505971262] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:27,924 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 07:09:27,927 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 07:09:27,927 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveShardLeaderReplica 07:09:27,927 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveShardLeaderReplica 07:09:27,929 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 07:09:27,929 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@2f283ffa} 07:09:27,929 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} started 07:09:27,930 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:27,930 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:27,930 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveShardLeaderReplica 07:09:27,931 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:27,933 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardLeaderReplica 07:09:27,934 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveShardLeaderReplica 07:09:27,935 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-2-shard-cars-config_testRemoveShardLeaderReplica, persistent : true 07:09:27,936 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testRemoveShardLeaderReplica with journal batch size 1 07:09:27,936 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-2-shard-people-config_testRemoveShardLeaderReplica, persistent : true 07:09:27,936 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 07:09:27,936 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-2-shard-people-config_testRemoveShardLeaderReplica with journal batch size 1 07:09:27,937 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica-notifier#-1585222002 created and ready for shard:member-2-shard-cars-config_testRemoveShardLeaderReplica 07:09:27,937 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica-notifier#-1017742777 created and ready for shard:member-2-shard-people-config_testRemoveShardLeaderReplica 07:09:27,937 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica/shard-member-2:snapshot-read#-1316953511] 07:09:27,937 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#899433166] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:27,938 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:27,938 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:27,938 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:27,938 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:27,938 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:09:27,938 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:27,938 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , received role change from null to Follower 07:09:27,939 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 07:09:27,939 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveShardLeaderReplica is initialized 07:09:27,939 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 07:09:27,939 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveShardLeaderReplica is initialized 07:09:27,943 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica/shard-member-2:snapshot-read#-1354759223] 07:09:27,943 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#2137188199] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:27,943 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica#340373389] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:27,943 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica-notifier#-1585222002] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:27,943 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica-notifier#-1017742777] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:27,943 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica/shard-member-2:snapshot-read#-1316953511] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:27,943 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica/shard-member-2:snapshot-read#-1354759223] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:27,943 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:09:27,943 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:27,943 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 07:09:27,943 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardLeaderReplica , received role change from null to Follower 07:09:27,943 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardLeaderReplica from null to Follower 07:09:27,943 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 07:09:27,943 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveShardLeaderReplica from null to Follower 07:09:27,944 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 07:09:27,945 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveShardLeaderReplica 07:09:27,946 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveShardLeaderReplica 07:09:27,947 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 07:09:27,948 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} started 07:09:27,948 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@49983b04} 07:09:27,948 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:27,948 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveShardLeaderReplica 07:09:27,949 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveShardLeaderReplica 07:09:27,949 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-2-shard-cars-oper_testRemoveShardLeaderReplica, persistent : true 07:09:27,949 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Shard created : member-2-shard-people-oper_testRemoveShardLeaderReplica, persistent : true 07:09:27,950 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testRemoveShardLeaderReplica with journal batch size 1 07:09:27,950 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testRemoveShardLeaderReplica with journal batch size 1 07:09:27,950 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica-notifier#-1918011641 created and ready for shard:member-2-shard-people-oper_testRemoveShardLeaderReplica 07:09:27,950 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica/shard-member-2:snapshot-read#-154579160] 07:09:27,950 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica#-1178039890] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:27,950 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica#-1335540482] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:27,950 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica-notifier#862422935 created and ready for shard:member-2-shard-cars-oper_testRemoveShardLeaderReplica 07:09:27,950 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica/shard-member-2:snapshot-read#1882970543] 07:09:27,953 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:27,953 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:27,953 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:09:27,953 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:27,953 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardLeaderReplica , received role change from null to Follower 07:09:27,953 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 07:09:27,953 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveShardLeaderReplica is initialized 07:09:27,953 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:27,953 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:27,953 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:09:27,953 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:27,953 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardLeaderReplica , received role change from null to Follower 07:09:27,953 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 07:09:27,953 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveShardLeaderReplica is initialized 07:09:27,953 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 07:09:27,953 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveShardLeaderReplica from null to Follower 07:09:27,953 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 07:09:27,953 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveShardLeaderReplica from null to Follower 07:09:27,964 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 07:09:27,971 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 07:09:27,973 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 07:09:27,973 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:27,973 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:27,974 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 07:09:27,974 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 07:09:27,974 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 07:09:27,974 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 07:09:27,974 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 07:09:27,974 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 07:09:27,974 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 07:09:27,974 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 07:09:27,974 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 07:09:27,974 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 07:09:27,974 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 07:09:27,974 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 07:09:27,985 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [9144782588357634485] 07:09:27,986 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 07:09:27,988 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:09:27,988 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 07:09:27,988 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:09:27,994 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:09:27,999 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:28,1 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:28,3 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardLeaderReplica 07:09:28,11 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-32897277] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:28,11 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 07:09:28,13 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 07:09:28,13 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testRemoveShardLeaderReplica 07:09:28,13 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testRemoveShardLeaderReplica 07:09:28,15 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 07:09:28,17 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} started 07:09:28,17 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@26682d69} 07:09:28,17 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:28,17 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testRemoveShardLeaderReplica 07:09:28,17 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:28,18 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:28,23 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardLeaderReplica 07:09:28,27 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testRemoveShardLeaderReplica 07:09:28,28 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-3-shard-cars-config_testRemoveShardLeaderReplica, persistent : true 07:09:28,29 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testRemoveShardLeaderReplica with journal batch size 1 07:09:28,29 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-3-shard-people-config_testRemoveShardLeaderReplica, persistent : true 07:09:28,29 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-3-shard-people-config_testRemoveShardLeaderReplica with journal batch size 1 07:09:28,30 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 07:09:28,30 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#730240807] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:28,30 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica#1795365955] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:28,30 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#2132483701] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:28,30 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica-notifier#-803772193 created and ready for shard:member-3-shard-cars-config_testRemoveShardLeaderReplica 07:09:28,30 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica-notifier#-803772193] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:28,30 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica/shard-member-3:snapshot-read#-1101080878] 07:09:28,30 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica/shard-member-3:snapshot-read#-1101080878] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:28,31 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica/shard-member-3:snapshot-read#-112133025] 07:09:28,31 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 07:09:28,31 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica/shard-member-3:snapshot-read#-112133025] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:28,32 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 07:09:28,32 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica-notifier#794562885 created and ready for shard:member-3-shard-people-config_testRemoveShardLeaderReplica 07:09:28,32 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica-notifier#794562885] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:28,33 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:28,33 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 07:09:28,33 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 07:09:28,33 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 07:09:28,33 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 07:09:28,36 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:28,36 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:28,36 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:28,36 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:28,36 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:09:28,36 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:28,36 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 07:09:28,36 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 07:09:28,36 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testRemoveShardLeaderReplica is initialized 07:09:28,36 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 07:09:28,36 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testRemoveShardLeaderReplica is initialized 07:09:28,36 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , received role change from null to Follower 07:09:28,36 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 07:09:28,36 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardLeaderReplica from null to Follower 07:09:28,36 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 07:09:28,36 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testRemoveShardLeaderReplica 07:09:28,36 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:09:28,36 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:28,37 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardLeaderReplica , received role change from null to Follower 07:09:28,37 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 07:09:28,37 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 07:09:28,37 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testRemoveShardLeaderReplica from null to Follower 07:09:28,37 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testRemoveShardLeaderReplica 07:09:28,37 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:28,37 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 07:09:28,37 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 07:09:28,57 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 07:09:28,58 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@3cf0ded7} 07:09:28,58 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} started 07:09:28,58 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 07:09:28,59 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:28,59 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testRemoveShardLeaderReplica 07:09:28,60 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 07:09:28,60 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 07:09:28,60 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica on shard member-3-shard-cars-oper_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica#1049500822] 07:09:28,60 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testRemoveShardLeaderReplica 07:09:28,60 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 07:09:28,60 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Shard created : member-3-shard-people-oper_testRemoveShardLeaderReplica, persistent : true 07:09:28,60 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Shard created : member-3-shard-cars-oper_testRemoveShardLeaderReplica, persistent : true 07:09:28,62 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testRemoveShardLeaderReplica with journal batch size 1 07:09:28,62 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica-notifier#1164042595 created and ready for shard:member-3-shard-people-oper_testRemoveShardLeaderReplica 07:09:28,62 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testRemoveShardLeaderReplica with journal batch size 1 07:09:28,62 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica/shard-member-3:snapshot-read#1296115042] 07:09:28,61 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 07:09:28,62 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica-notifier#-1686542467 created and ready for shard:member-3-shard-cars-oper_testRemoveShardLeaderReplica 07:09:28,62 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica/shard-member-3:snapshot-read#1063033881] 07:09:28,62 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica on shard member-3-shard-people-oper_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica#37214100] 07:09:28,62 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica#1049500822] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:28,62 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica#37214100] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:28,63 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:28,63 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:28,63 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:28,63 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:09:28,63 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:28,63 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:28,63 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardLeaderReplica , received role change from null to Follower 07:09:28,63 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 07:09:28,63 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:09:28,64 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 07:09:28,64 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:28,64 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testRemoveShardLeaderReplica is initialized 07:09:28,64 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 07:09:28,64 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testRemoveShardLeaderReplica is initialized 07:09:28,64 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardLeaderReplica , received role change from null to Follower 07:09:28,64 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 07:09:28,64 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testRemoveShardLeaderReplica from null to Follower 07:09:28,64 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 07:09:28,64 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 07:09:28,64 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testRemoveShardLeaderReplica from null to Follower 07:09:28,212 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 07:09:28,212 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Follower): Received ElectionTimeout - switching to Candidate 07:09:28,212 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testRemoveShardLeaderReplica, member-3-shard-cars-config_testRemoveShardLeaderReplica] 07:09:28,212 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 07:09:28,212 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:09:28,212 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:09:28,212 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:28,213 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 07:09:28,213 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from Follower to Candidate 07:09:28,272 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 07:09:28,272 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Follower): Received ElectionTimeout - switching to Candidate 07:09:28,272 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Election: Candidate has following voting peers: [member-3-shard-people-oper_testRemoveShardLeaderReplica, member-2-shard-people-oper_testRemoveShardLeaderReplica] 07:09:28,272 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 07:09:28,272 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:09:28,273 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:09:28,273 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:28,273 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Follower to Candidate 07:09:28,273 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Follower to Candidate 07:09:28,282 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 07:09:28,282 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Follower): Received ElectionTimeout - switching to Candidate 07:09:28,282 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Election: Candidate has following voting peers: [member-3-shard-people-config_testRemoveShardLeaderReplica, member-2-shard-people-config_testRemoveShardLeaderReplica] 07:09:28,282 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 07:09:28,282 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:09:28,282 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:09:28,282 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:28,283 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 07:09:28,283 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from Follower to Candidate 07:09:28,302 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 07:09:28,302 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Received ElectionTimeout - switching to Candidate 07:09:28,302 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-oper_testRemoveShardLeaderReplica, member-2-shard-cars-oper_testRemoveShardLeaderReplica] 07:09:28,303 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 07:09:28,303 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:09:28,303 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:09:28,303 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:28,303 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Follower to Candidate 07:09:28,303 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Follower to Candidate 07:09:28,592 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 07:09:28,592 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 07:09:28,612 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 07:09:28,612 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 07:09:28,632 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 07:09:28,632 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 07:09:28,642 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 07:09:28,642 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 07:09:28,863 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 07:09:28,863 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 07:09:28,864 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:28,864 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 07:09:28,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 07:09:28,864 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 07:09:28,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 07:09:28,864 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:09:28,864 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 07:09:28,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 07:09:28,864 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 07:09:28,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 07:09:28,864 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 07:09:28,864 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 07:09:28,864 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 07:09:28,864 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-2-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 07:09:28,864 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:28,864 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 07:09:28,864 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 07:09:28,864 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 07:09:28,864 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 07:09:28,864 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 07:09:28,864 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 07:09:28,864 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:09:28,864 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 07:09:28,864 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 07:09:28,864 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 07:09:28,864 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 07:09:28,864 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 07:09:28,864 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 07:09:28,871 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:28,871 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 07:09:28,871 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 07:09:28,871 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 07:09:28,871 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 07:09:28,871 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:09:28,871 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 07:09:28,871 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 07:09:28,871 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 07:09:28,871 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 07:09:28,871 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 07:09:28,871 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 07:09:28,871 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:28,871 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 07:09:28,871 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 07:09:28,871 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 07:09:28,871 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 07:09:28,871 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 07:09:28,871 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:09:28,872 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 07:09:28,872 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 07:09:28,872 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 07:09:28,872 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 07:09:28,872 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 07:09:28,930 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:28,931 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:28,931 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 07:09:28,931 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 07:09:28,931 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 07:09:28,931 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 07:09:28,931 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:09:28,931 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 07:09:28,931 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 07:09:28,931 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 07:09:28,931 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 07:09:28,931 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 07:09:28,931 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 07:09:28,931 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-2-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 07:09:28,931 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 07:09:28,931 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 07:09:28,931 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 07:09:28,931 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:09:28,931 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 07:09:28,931 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 07:09:28,931 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 07:09:28,931 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 07:09:28,932 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 07:09:28,932 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 07:09:28,972 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 07:09:28,972 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 07:09:28,973 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1} to peer member-3-shard-people-config_testRemoveShardLeaderReplica 07:09:28,973 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1} to peer member-2-shard-people-config_testRemoveShardLeaderReplica 07:09:28,974 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:28,974 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): Cluster state: [] 07:09:28,974 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:09:28,974 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:09:28,974 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:09:28,974 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:09:28,974 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:28,974 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): Cluster state: [] 07:09:28,974 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:09:28,974 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:09:28,974 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:09:28,975 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:09:28,975 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): RaftRPC message received RequestVoteReply{term=3, voteGranted=true}, my term is 3 07:09:28,975 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): handleRequestVoteReply: RequestVoteReply{term=3, voteGranted=true}, current voteCount: 1 07:09:28,975 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-3-shard-people-config_testRemoveShardLeaderReplica, member-2-shard-people-config_testRemoveShardLeaderReplica] 07:09:28,975 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:28,975 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:28,975 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 07:09:28,975 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 07:09:28,975 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testRemoveShardLeaderReplica, leaderId=member-1-shard-people-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 07:09:28,976 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 07:09:28,976 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 07:09:28,976 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from Candidate to Leader 07:09:28,976 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardLeaderReplica status sync done false 07:09:28,976 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testRemoveShardLeaderReplica, leaderId=member-1-shard-people-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 07:09:28,977 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 07:09:28,977 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:28,977 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testRemoveShardLeaderReplica, leaderId=member-1-shard-people-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 07:09:28,977 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveShardLeaderReplica status sync done false 07:09:28,977 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 07:09:28,978 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:29,3 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 07:09:29,12 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 07:09:29,12 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 07:09:29,12 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1} to peer member-2-shard-cars-config_testRemoveShardLeaderReplica 07:09:29,12 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1} to peer member-3-shard-cars-config_testRemoveShardLeaderReplica 07:09:29,13 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:29,13 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Cluster state: [] 07:09:29,13 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:29,13 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:09:29,13 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:09:29,13 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Cluster state: [] 07:09:29,13 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:09:29,13 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:09:29,13 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:09:29,13 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:09:29,13 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:09:29,13 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:09:29,14 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): RaftRPC message received RequestVoteReply{term=3, voteGranted=true}, my term is 3 07:09:29,14 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): handleRequestVoteReply: RequestVoteReply{term=3, voteGranted=true}, current voteCount: 1 07:09:29,14 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testRemoveShardLeaderReplica, member-3-shard-cars-config_testRemoveShardLeaderReplica] 07:09:29,14 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:29,14 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:29,14 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 07:09:29,14 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 07:09:29,14 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testRemoveShardLeaderReplica, leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 07:09:29,15 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 07:09:29,15 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 07:09:29,15 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from Candidate to Leader 07:09:29,15 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 07:09:29,15 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} is now ready 07:09:29,15 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-1698317565] 07:09:29,15 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:09:29,15 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-372822596] 07:09:29,15 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-372822596] 07:09:29,15 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-505971262] 07:09:29,16 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done false 07:09:29,16 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 07:09:29,16 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardLeaderReplica status sync done false 07:09:29,16 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#899433166] 07:09:29,16 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 07:09:29,16 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#899433166] 07:09:29,16 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveShardLeaderReplica, leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 07:09:29,16 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 07:09:29,16 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 07:09:29,16 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-32897277] 07:09:29,16 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 07:09:29,16 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#730240807] 07:09:29,16 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#730240807] 07:09:29,16 AM [main] [INFO] ClusterAdminRpcService - Removing replica for shard cars memberName member-1, datastoreType Config 07:09:29,17 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 07:09:29,17 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:29,17 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 07:09:29,17 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:29,16 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-1}] 07:09:29,17 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 07:09:29,17 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: LocalPrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-372822596, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@282682cb] 07:09:29,18 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-372822596 for shard member-1-shard-cars-config_testRemoveShardLeaderReplica 07:09:29,18 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onRemoveServer: RemoveServer{serverId='member-1-shard-cars-config_testRemoveShardLeaderReplica'}, state: Idle 07:09:29,18 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]] 07:09:29,18 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 07:09:29,18 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 07:09:29,18 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=c6aafa85-0f5b-48b8-8961-25f8645690b1}, logIndex: 0, isSendImmediate: true 07:09:29,18 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:29,18 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testRemoveShardLeaderReplica 07:09:29,18 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testRemoveShardLeaderReplica: AppendEntries{term=3, leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]]]} 07:09:29,18 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:29,18 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-config_testRemoveShardLeaderReplica 07:09:29,18 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testRemoveShardLeaderReplica: AppendEntries{term=3, leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]]]} 07:09:29,18 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: Returning OK for operation RemoveServer{serverId='member-1-shard-cars-config_testRemoveShardLeaderReplica'} 07:09:29,18 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveShardLeaderReplica} for shard cars 07:09:29,18 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Leader shard successfully removed the replica shard cars 07:09:29,18 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] ClusterAdminRpcService - Successfully removed replica for shard cars 07:09:29,19 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 07:09:29,19 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-505971262] 07:09:29,19 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]]]} 07:09:29,19 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]]]} 07:09:29,19 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Number of entries to be appended = 1 07:09:29,19 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Number of entries to be appended = 1 07:09:29,19 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 07:09:29,19 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 07:09:29,19 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]] 07:09:29,19 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]] 07:09:29,19 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 07:09:29,19 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 07:09:29,19 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testRemoveShardLeaderReplica, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica, votingState=VOTING]] 07:09:29,19 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Log size is now 1 07:09:29,19 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-config_testRemoveShardLeaderReplica, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica, votingState=VOTING]] 07:09:29,19 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Log size is now 1 07:09:29,19 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:29,20 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:29,20 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 07:09:29,20 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#899433166] 07:09:29,20 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#899433166] 07:09:29,20 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 07:09:29,20 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 07:09:29,20 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done true 07:09:29,20 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardLeaderReplica status sync done true 07:09:29,20 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testRemoveShardLeaderReplica updated: matchIndex: 0, nextIndex: 1 07:09:29,20 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 07:09:29,20 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Setting last applied to 0 07:09:29,20 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Moving last applied index from -1 to 0 07:09:29,20 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]] 07:09:29,20 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:29,20 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]] has been successfully replicated to a majority of followers 07:09:29,21 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Persisting ApplyJournalEntries with index=0 07:09:29,21 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testRemoveShardLeaderReplica updated: matchIndex: 0, nextIndex: 1 07:09:29,21 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:29,21 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-372822596] with 10000 ms timeout 07:09:29,21 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1 : Local Shard replica for shard cars has been removed 07:09:29,21 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: persisting the shard list [people] 07:09:29,21 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 07:09:29,21 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Successfully deleted prior snapshots 07:09:29,21 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onShutDown 07:09:29,21 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Initiating leader transfer 07:09:29,22 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@348916de 07:09:29,22 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-config_testRemoveShardLeaderReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@348916de 07:09:29,22 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: pauseLeader successfully completed - doing transfer 07:09:29,22 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Attempting to transfer leadership 07:09:29,22 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Commit index set to 0 07:09:29,22 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:29,22 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 07:09:29,22 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:29,22 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Setting last applied to 0 07:09:29,22 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Moving last applied index from -1 to 0 07:09:29,22 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]] 07:09:29,22 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 07:09:29,22 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Persisting ApplyJournalEntries with index=0 07:09:29,22 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 07:09:29,22 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1} to peer member-3-shard-cars-oper_testRemoveShardLeaderReplica 07:09:29,22 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1} to peer member-2-shard-cars-oper_testRemoveShardLeaderReplica 07:09:29,22 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=13] 07:09:29,22 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - No shard found with member Id member-1-shard-cars-config_testRemoveShardLeaderReplica 07:09:29,23 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-config_testRemoveShardLeaderReplica, matchIndex: 0, lastIndex: 0, isVoting: true 07:09:29,23 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Follower's log matches - sending ElectionTimeout 07:09:29,23 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:29,23 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:29,23 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Leader transfer complete 07:09:29,23 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: leader transfer complete - waiting for new leader 07:09:29,23 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica] 07:09:29,24 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=13] 07:09:29,24 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Commit index set to 0 07:09:29,24 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 07:09:29,24 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Setting last applied to 0 07:09:29,24 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Moving last applied index from -1 to 0 07:09:29,24 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]] 07:09:29,24 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Persisting ApplyJournalEntries with index=0 07:09:29,24 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica] 07:09:29,24 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=13] 07:09:29,25 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:29,25 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Cluster state: [] 07:09:29,25 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:09:29,25 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:09:29,25 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:09:29,25 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:09:29,25 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:29,25 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Cluster state: [] 07:09:29,25 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:09:29,25 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:09:29,25 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:09:29,25 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:09:29,25 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): RaftRPC message received RequestVoteReply{term=3, voteGranted=true}, my term is 3 07:09:29,25 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): handleRequestVoteReply: RequestVoteReply{term=3, voteGranted=true}, current voteCount: 1 07:09:29,26 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardLeaderReplica, member-2-shard-cars-oper_testRemoveShardLeaderReplica] 07:09:29,26 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:29,26 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:29,26 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 07:09:29,26 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 07:09:29,26 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: became leader with frontend state for [] 07:09:29,26 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Candidate to Leader 07:09:29,26 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, leaderId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 07:09:29,26 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Candidate to Leader 07:09:29,26 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Received TimeoutNow - switching to Candidate 07:09:29,26 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-config_testRemoveShardLeaderReplica] 07:09:29,26 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 4 07:09:29,26 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Sending RequestVote{term=4, candidateId=member-2-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3} to peer member-3-shard-cars-config_testRemoveShardLeaderReplica 07:09:29,26 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 4 07:09:29,26 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:09:29,26 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:29,26 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 07:09:29,26 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=-1] 07:09:29,26 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardLeaderReplica from Follower to Candidate 07:09:29,27 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testRemoveShardLeaderReplica, leaderId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 07:09:29,27 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveShardLeaderReplica status sync done false 07:09:29,27 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:29,27 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Cluster state: [] 07:09:29,27 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 07:09:29,27 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote{term=4, candidateId=member-2-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3} with higher term appears reachable, updating term. 07:09:29,27 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Term 4 in "RequestVote{term=4, candidateId=member-2-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3}" message is greater than follower's term 3 - updating term 07:09:29,27 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:29,27 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote{term=4, candidateId=member-2-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3} - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 3 07:09:29,27 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply{term=4, voteGranted=true} 07:09:29,27 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, leaderId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 07:09:29,27 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardLeaderReplica status sync done false 07:09:29,28 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate): RaftRPC message received RequestVoteReply{term=4, voteGranted=true}, my term is 4 07:09:29,28 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate): handleRequestVoteReply: RequestVoteReply{term=4, voteGranted=true}, current voteCount: 1 07:09:29,28 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-3-shard-cars-config_testRemoveShardLeaderReplica] 07:09:29,28 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:29,28 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 07:09:29,28 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 07:09:29,28 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 07:09:29,28 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 07:09:29,28 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 07:09:29,28 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardLeaderReplica from Candidate to Leader 07:09:29,28 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 07:09:29,28 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done false 07:09:29,29 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveShardLeaderReplica, leaderId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 07:09:29,29 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 07:09:29,29 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 07:09:29,29 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:29,29 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testRemoveShardLeaderReplica updated: matchIndex: 0, nextIndex: 1 07:09:29,29 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:29,30 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done true 07:09:29,42 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 07:09:29,42 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 07:09:29,42 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1} to peer member-3-shard-people-oper_testRemoveShardLeaderReplica 07:09:29,42 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1} to peer member-2-shard-people-oper_testRemoveShardLeaderReplica 07:09:29,43 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:29,43 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): Cluster state: [] 07:09:29,43 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:09:29,43 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:09:29,43 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:29,43 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:09:29,43 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): Cluster state: [] 07:09:29,43 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:09:29,43 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:09:29,43 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:09:29,43 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:09:29,43 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:09:29,44 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): RaftRPC message received RequestVoteReply{term=3, voteGranted=true}, my term is 3 07:09:29,44 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): handleRequestVoteReply: RequestVoteReply{term=3, voteGranted=true}, current voteCount: 1 07:09:29,44 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-3-shard-people-oper_testRemoveShardLeaderReplica, member-2-shard-people-oper_testRemoveShardLeaderReplica] 07:09:29,44 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:29,44 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:29,44 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 07:09:29,44 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 07:09:29,44 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: became leader with frontend state for [] 07:09:29,44 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testRemoveShardLeaderReplica, leaderId=member-1-shard-people-oper_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 07:09:29,45 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Candidate to Leader 07:09:29,45 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Candidate to Leader 07:09:29,45 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 07:09:29,45 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveShardLeaderReplica status sync done false 07:09:29,45 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testRemoveShardLeaderReplica, leaderId=member-1-shard-people-oper_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 07:09:29,45 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 07:09:29,45 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testRemoveShardLeaderReplica, leaderId=member-1-shard-people-oper_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 07:09:29,46 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 07:09:29,46 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardLeaderReplica status sync done false 07:09:29,46 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 07:09:29,46 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:29,46 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 07:09:29,46 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:29,73 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#899433166] 07:09:29,73 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-505971262] 07:09:29,74 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 07:09:29,74 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#899433166] 07:09:29,74 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#899433166] 07:09:29,74 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-32897277] 07:09:29,74 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 07:09:29,74 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#730240807] 07:09:29,74 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#730240807] 07:09:29,74 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-1698317565] 07:09:29,74 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:09:29,74 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Local shard cars not found - shards present: [people] 07:09:29,75 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 07:09:29,75 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testRemoveShardLeaderReplica 07:09:29,75 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testRemoveShardLeaderReplica 07:09:29,75 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 07:09:29,75 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onShutDown 07:09:29,75 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: onShutDown 07:09:29,75 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testRemoveShardLeaderReplica 07:09:29,75 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-3-shard-people-config_testRemoveShardLeaderReplica 07:09:29,76 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 07:09:29,76 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 07:09:29,76 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 07:09:29,77 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testRemoveShardLeaderReplica 07:09:29,77 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testRemoveShardLeaderReplica 07:09:29,77 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 07:09:29,77 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: onShutDown 07:09:29,77 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: onShutDown 07:09:29,77 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testRemoveShardLeaderReplica 07:09:29,78 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testRemoveShardLeaderReplica 07:09:29,78 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 07:09:29,78 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 07:09:29,80 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:09:29,80 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:09:29,85 AM [cluster-test-pekko.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:29,85 AM [cluster-test-pekko.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:29,86 AM [cluster-test-pekko.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:29,86 AM [cluster-test-pekko.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:29,88 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 07:09:29,93 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 07:09:29,94 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testRemoveShardLeaderReplica 07:09:29,94 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveShardLeaderReplica 07:09:29,94 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 07:09:29,94 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onShutDown 07:09:29,94 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Initiating leader transfer 07:09:29,94 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@662d67c 07:09:29,94 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-config_testRemoveShardLeaderReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@662d67c 07:09:29,94 AM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testRemoveShardLeaderReplica: pauseLeader successfully completed - doing transfer 07:09:29,94 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Attempting to transfer leadership 07:09:29,94 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:29,94 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: onShutDown 07:09:29,94 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=13] 07:09:29,94 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-2-shard-people-config_testRemoveShardLeaderReplica 07:09:29,333 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Cannot append entries because sender's term 3 is less than 4 07:09:29,334 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Term 4 in "AppendEntriesReply{term=4, success=false, followerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}" message is greater than leader's term 3 - switching to Follower 07:09:29,334 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to Follower, election term: 4 07:09:29,334 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 07:09:29,334 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:29,334 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from Leader to Follower 07:09:29,334 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=-1] 07:09:29,334 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - No shard found with member Id member-1-shard-cars-config_testRemoveShardLeaderReplica 07:09:29,334 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from Leader to Follower 07:09:29,343 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardLeaderReplica status sync done true 07:09:29,343 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 314, lastApplied : -1, commitIndex : -1 07:09:29,363 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardLeaderReplica status sync done true 07:09:29,364 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 07:09:29,665 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 07:09:29,685 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:09:29,882 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 07:09:29,985 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:09:30,5 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:09:30,92 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [9144782588357634485] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 07:09:30,95 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [9144782588357634485] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 07:09:30,304 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:09:30,323 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 07:09:30,625 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:09:30,644 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:09:30,944 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:09:30,963 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:09:31,42 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: leader not elected in time 07:09:31,42 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: Successfully transferred leadership to null in 2.020 s 07:09:31,42 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: leader transfer succeeded - sending PoisonPill 07:09:31,42 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testRemoveShardLeaderReplica 07:09:31,43 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-372822596] 07:09:31,265 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:09:31,284 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 07:09:31,585 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:09:31,604 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:09:31,905 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:09:31,926 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 07:09:32,224 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:09:32,244 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 07:09:32,545 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 07:09:32,565 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:09:32,865 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:09:32,884 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:09:33,184 AM [cluster-test-shard-dispatcher-15] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:09:33,204 AM [cluster-test-shard-dispatcher-15] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:09:33,504 AM [cluster-test-shard-dispatcher-15] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:09:33,524 AM [cluster-test-shard-dispatcher-15] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:09:33,824 AM [cluster-test-shard-dispatcher-15] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:09:33,845 AM [cluster-test-shard-dispatcher-15] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:09:34,92 AM [cluster-test-shard-dispatcher-15] [WARN] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: At least 1 followers need to be active, Switching member-1-shard-people-config_testRemoveShardLeaderReplica from Leader to IsolatedLeader 07:09:34,92 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-people-config_testRemoveShardLeaderReplica, member-2-shard-people-config_testRemoveShardLeaderReplica] 07:09:34,92 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:34,92 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:34,93 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 07:09:34,93 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 07:09:34,93 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 07:09:34,93 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from Leader to IsolatedLeader 07:09:34,142 AM [cluster-test-shard-dispatcher-15] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: At least 1 followers need to be active, Switching member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 07:09:34,142 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardLeaderReplica, member-2-shard-cars-oper_testRemoveShardLeaderReplica] 07:09:34,142 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:34,142 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:34,142 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 07:09:34,142 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 07:09:34,142 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 07:09:34,142 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 07:09:34,144 AM [cluster-test-shard-dispatcher-15] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 07:09:34,144 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - IsolatedLeader member-1-shard-cars-oper_testRemoveShardLeaderReplica switching from IsolatedLeader to Leader 07:09:34,144 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardLeaderReplica, member-2-shard-cars-oper_testRemoveShardLeaderReplica] 07:09:34,144 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:34,144 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:34,144 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 07:09:34,144 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 07:09:34,145 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from IsolatedLeader to Leader 07:09:34,145 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from IsolatedLeader to Leader 07:09:34,145 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 07:09:34,162 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: At least 1 followers need to be active, Switching member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 07:09:34,162 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-people-oper_testRemoveShardLeaderReplica, member-2-shard-people-oper_testRemoveShardLeaderReplica] 07:09:34,162 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:34,162 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:34,162 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 07:09:34,162 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 07:09:34,162 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 07:09:34,162 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 07:09:34,164 AM [cluster-test-shard-dispatcher-15] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:09:34,165 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - IsolatedLeader member-1-shard-people-oper_testRemoveShardLeaderReplica switching from IsolatedLeader to Leader 07:09:34,165 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-3-shard-people-oper_testRemoveShardLeaderReplica, member-2-shard-people-oper_testRemoveShardLeaderReplica] 07:09:34,165 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:34,165 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:34,165 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 07:09:34,165 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 07:09:34,165 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from IsolatedLeader to Leader 07:09:34,165 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from IsolatedLeader to Leader 07:09:34,165 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 07:09:34,465 AM [cluster-test-shard-dispatcher-15] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 07:09:34,484 AM [cluster-test-shard-dispatcher-15] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 07:09:34,785 AM [cluster-test-shard-dispatcher-15] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:09:34,804 AM [cluster-test-shard-dispatcher-15] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:09:34,973 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Up)]. 07:09:34,974 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:34,974 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:34,975 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 07:09:35,36 AM [cluster-test-pekko.actor.default-dispatcher-11] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Up)]. 07:09:35,36 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:35,36 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:35,105 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:09:35,124 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:09:35,424 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:09:35,444 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:09:35,744 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:09:35,764 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:09:36,64 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:09:36,84 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:09:36,385 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:09:36,404 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:09:36,658 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 07:09:36,704 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:09:36,709 AM [cluster-test-pekko.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 07:09:36,724 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:09:37,25 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:09:37,44 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:09:37,344 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 07:09:37,364 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 07:09:37,664 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:09:37,695 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 331, lastApplied : -1, commitIndex : -1 07:09:37,985 AM [cluster-test-shard-dispatcher-15] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 07:09:38,3 AM [cluster-test-shard-dispatcher-15] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 307, lastApplied : -1, commitIndex : -1 07:09:38,304 AM [cluster-test-shard-dispatcher-15] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 07:09:38,305 AM [cluster-test-shard-dispatcher-26] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testRemoveShardLeaderReplica: leader transfer aborted 07:09:38,305 AM [cluster-test-shard-dispatcher-26] [WARN] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testRemoveShardLeaderReplica: Failed to transfer leadership in 9.211 s 07:09:38,305 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: In unpauseLeader 07:09:38,305 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: leader transfer failed - sending PoisonPill 07:09:38,305 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Leadership transfer expired 07:09:38,306 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testRemoveShardLeaderReplica 07:09:38,307 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 07:09:38,307 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 07:09:38,308 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 07:09:38,308 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveShardLeaderReplica 07:09:38,308 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveShardLeaderReplica 07:09:38,308 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 07:09:38,308 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: onShutDown 07:09:38,308 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: onShutDown 07:09:38,309 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testRemoveShardLeaderReplica 07:09:38,309 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testRemoveShardLeaderReplica 07:09:38,309 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 07:09:38,309 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 07:09:38,311 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:09:38,311 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:09:39,262 AM [cluster-test-shard-dispatcher-15] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: At least 1 followers need to be active, Switching member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 07:09:39,263 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardLeaderReplica, member-2-shard-cars-oper_testRemoveShardLeaderReplica] 07:09:39,263 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:39,263 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:39,263 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 07:09:39,263 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 07:09:39,263 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 07:09:39,264 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 07:09:39,282 AM [cluster-test-shard-dispatcher-15] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: At least 1 followers need to be active, Switching member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 07:09:39,283 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-people-oper_testRemoveShardLeaderReplica, member-2-shard-people-oper_testRemoveShardLeaderReplica] 07:09:39,283 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:39,283 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:39,283 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 07:09:39,283 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 07:09:39,283 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 07:09:39,283 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 07:09:39,328 AM [cluster-test-pekko.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:39,329 AM [cluster-test-pekko.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:39,330 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 07:09:39,332 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-7040090376830374142] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated [WARN] [01/31/2025 07:09:39.332] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:09:39,336 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 07:09:39,336 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testRemoveShardLeaderReplica 07:09:39,336 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 07:09:39,336 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: onShutDown 07:09:39,337 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-people-config_testRemoveShardLeaderReplica 07:09:39,337 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 07:09:39,337 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 07:09:39,338 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 07:09:39,338 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testRemoveShardLeaderReplica 07:09:39,338 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testRemoveShardLeaderReplica 07:09:39,338 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 07:09:39,338 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onShutDown 07:09:39,338 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: onShutDown 07:09:39,343 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testRemoveShardLeaderReplica 07:09:39,343 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testRemoveShardLeaderReplica 07:09:39,343 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 07:09:39,343 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 07:09:39,344 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:09:39,344 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:09:39,346 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [01/31/2025 07:09:39.348] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:09:39,367 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 07:09:39,375 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [8163926250907788058] 07:09:39,377 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 07:09:39,380 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 07:09:39,380 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 07:09:39,382 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:09:39,382 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:09:39,394 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:39,396 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:39,399 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 07:09:39,399 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 07:09:39,399 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,399 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 07:09:39,404 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 07:09:39,406 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard#-1211618427] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,408 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 07:09:39,408 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,409 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,409 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:39,409 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,409 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,426 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 07:09:39,427 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@3f0908e1} 07:09:39,427 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} started 07:09:39,428 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:39,429 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:39,431 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:39,431 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,431 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,431 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,431 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,431 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#-279972093] 07:09:39,431 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,432 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,432 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,432 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard#1787536737] 07:09:39,432 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard, persistent : true 07:09:39,432 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard, persistent : true 07:09:39,432 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with journal batch size 1 07:09:39,433 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with journal batch size 1 07:09:39,433 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#-279972093] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,433 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#-1070703011 created and ready for shard:member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,434 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 07:09:39,434 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard#1787536737] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,434 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#-857860065 created and ready for shard:member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,435 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard/shard-member-1:snapshot-read#782395155] 07:09:39,435 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:39,435 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:39,435 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 07:09:39,435 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 07:09:39,435 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard is initialized 07:09:39,435 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:39,435 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Follower): Received TimeoutNow - switching to Candidate 07:09:39,436 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Election: Candidate has following voting peers: [] 07:09:39,436 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 07:09:39,436 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:09:39,436 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 07:09:39,436 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:39,436 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:39,436 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:39,436 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 07:09:39,436 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:39,436 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Follower): Received TimeoutNow - switching to Candidate 07:09:39,436 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Election: Candidate has following voting peers: [] 07:09:39,436 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 07:09:39,436 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:09:39,436 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 07:09:39,436 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:39,437 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 07:09:39,437 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard is initialized 07:09:39,437 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 07:09:39,437 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,437 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Received ElectionTimeout 07:09:39,437 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Leader): Election: Leader has following peers: [] 07:09:39,437 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 07:09:39,437 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 07:09:39,437 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,437 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 07:09:39,437 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Received ElectionTimeout 07:09:39,437 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:39,437 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,437 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,437 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Leader): Election: Leader has following peers: [] 07:09:39,437 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 07:09:39,437 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 07:09:39,437 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 07:09:39,437 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 07:09:39,437 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 07:09:39,437 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 07:09:39,438 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,438 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard, leaderPayloadVersion=13] 07:09:39,438 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 07:09:39,438 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 07:09:39,438 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 07:09:39,438 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 07:09:39,438 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,438 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard, leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard, leaderPayloadVersion=13] 07:09:39,438 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 07:09:39,438 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testChangeMemberVotingStatesForSingleNodeShard is ready 07:09:39,439 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#-1070703011] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,439 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard#-2096699687] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,439 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#-857860065] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,439 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard/shard-member-1:snapshot-read#782395155] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,439 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard/shard-member-1:snapshot-read#-1837618344] 07:09:39,439 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard/shard-member-1:snapshot-read#-1837618344] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,443 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 07:09:39,443 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@6409e69b} 07:09:39,444 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard, generation=0} started 07:09:39,444 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:39,444 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,444 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 07:09:39,444 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} is now ready 07:09:39,444 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,444 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,444 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard#-1284314797] 07:09:39,444 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,444 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,444 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,444 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard#1942648701] 07:09:39,445 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard, persistent : true 07:09:39,445 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard, persistent : true 07:09:39,445 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with journal batch size 1 07:09:39,445 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard#1942648701] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,445 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard-notifier#-841657713 created and ready for shard:member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,445 AM [main] [INFO] ClusterAdminRpcService - Change member voting states for shard cars: {member-1=false} 07:09:39,446 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard/shard-member-1:snapshot-read#-1340307098] 07:09:39,446 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with journal batch size 1 07:09:39,446 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-1=false}] 07:09:39,446 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard-notifier#1605822102 created and ready for shard:member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,446 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard/shard-member-1:snapshot-read#-1271642198] 07:09:39,446 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:09:39,446 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,447 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard=false}, serversVisited=[]], state: Idle 07:09:39,447 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 07:09:39,447 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Returning INVALID_REQUEST for operation ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard=false}, serversVisited=[]] 07:09:39,447 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=INVALID_REQUEST, leaderHint=member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard} from local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,447 AM [cluster-test-pekko.actor.default-dispatcher-6] [ERROR] ClusterAdminRpcService - Failed to change member voting states for shard cars java.lang.IllegalArgumentException: The requested voting state change for shard cars is invalid. At least one member must be voting at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager$11.onComplete(ShardManager.java:1561) at org.apache.pekko.dispatch.OnComplete.internal(Future.scala:310) at org.apache.pekko.dispatch.OnComplete.internal(Future.scala:307) at org.apache.pekko.dispatch.japi$CallbackBridge.apply(Future.scala:234) at org.apache.pekko.dispatch.japi$CallbackBridge.apply(Future.scala:231) at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:484) at org.apache.pekko.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:73) at org.apache.pekko.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:110) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18) at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94) at org.apache.pekko.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:110) at org.apache.pekko.dispatch.TaskInvocation.run(AbstractDispatcher.scala:59) at org.apache.pekko.dispatch.ForkJoinExecutorConfigurator$PekkoForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:57) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 07:09:39,447 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard#-1211618427] 07:09:39,447 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:09:39,447 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#-279972093] 07:09:39,447 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#-279972093] 07:09:39,448 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 07:09:39,448 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:39,448 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:39,448 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:39,448 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:39,448 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,448 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 07:09:39,448 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:39,448 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Follower): Received TimeoutNow - switching to Candidate 07:09:39,448 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Election: Candidate has following voting peers: [] 07:09:39,448 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 07:09:39,448 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:09:39,448 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 07:09:39,448 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:39,448 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 07:09:39,448 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard is initialized 07:09:39,448 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 07:09:39,448 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard is initialized 07:09:39,448 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 07:09:39,448 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 07:09:39,449 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard#-1284314797] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,451 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 07:09:39,451 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:39,451 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 07:09:39,451 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Follower): Received TimeoutNow - switching to Candidate 07:09:39,451 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Election: Candidate has following voting peers: [] 07:09:39,451 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 07:09:39,451 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:09:39,451 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 07:09:39,451 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:39,451 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 07:09:39,451 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Received ElectionTimeout 07:09:39,451 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Leader): Election: Leader has following peers: [] 07:09:39,451 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 07:09:39,451 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 07:09:39,451 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 07:09:39,451 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 07:09:39,451 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,451 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard, leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard, leaderPayloadVersion=13] 07:09:39,452 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 07:09:39,452 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Received ElectionTimeout 07:09:39,452 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Leader): Election: Leader has following peers: [] 07:09:39,452 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 07:09:39,452 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 07:09:39,452 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 07:09:39,452 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 07:09:39,452 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,452 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard, leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard, leaderPayloadVersion=13] 07:09:39,452 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 07:09:39,452 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testChangeMemberVotingStatesForSingleNodeShard is ready 07:09:39,452 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,452 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onShutDown 07:09:39,452 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 07:09:39,452 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onShutDown 07:09:39,452 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@74aabb4 07:09:39,452 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@74aabb4 07:09:39,453 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,453 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@2eb0591d 07:09:39,454 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@2eb0591d 07:09:39,454 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,454 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 07:09:39,454 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 07:09:39,454 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 07:09:39,454 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,454 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onShutDown 07:09:39,454 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@2077b2bb 07:09:39,454 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@2077b2bb 07:09:39,455 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,455 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,455 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onShutDown 07:09:39,455 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@2448326d 07:09:39,455 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@2448326d 07:09:39,455 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 07:09:39,455 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 07:09:39,455 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 07:09:39,455 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 07:09:39,457 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:09:39,457 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:09:39,472 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:09:39,483 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [1733387355066602096] 07:09:39,484 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 07:09:39,486 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 07:09:39,486 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 07:09:39,487 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:09:39,487 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:09:39,495 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:39,496 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 07:09:39,496 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 07:09:39,496 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:39,497 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 07:09:39,498 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatusForShard 07:09:39,504 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 07:09:39,505 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#1840577073] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,506 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 07:09:39,506 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:39,507 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testChangeMemberVotingStatusForShard 07:09:39,507 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:39,507 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:39,507 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 07:09:39,510 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 07:09:39,510 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@37bb7f07} 07:09:39,511 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} started 07:09:39,511 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:39,512 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:39,514 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatusForShard 07:09:39,515 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:39,515 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:39,515 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:39,515 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:39,515 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard on shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-59003840] 07:09:39,515 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testChangeMemberVotingStatusForShard 07:09:39,515 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-config_testChangeMemberVotingStatusForShard, persistent : true 07:09:39,515 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-59003840] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,516 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 07:09:39,516 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 07:09:39,516 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard on shard member-1-shard-people-config_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard#1490933299] 07:09:39,516 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-people-config_testChangeMemberVotingStatusForShard, persistent : true 07:09:39,516 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#-764231926 created and ready for shard:member-1-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:39,516 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard/shard-member-1:snapshot-read#1932153255] 07:09:39,516 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testChangeMemberVotingStatusForShard with journal batch size 1 07:09:39,516 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard#1490933299] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,516 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#-764231926] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,516 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard/shard-member-1:snapshot-read#1932153255] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,516 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-2099314541] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,516 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#493936475] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,516 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard/shard-member-1:snapshot-read#843585693] 07:09:39,517 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard/shard-member-1:snapshot-read#843585693] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,517 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 07:09:39,517 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:39,517 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:39,517 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-people-config_testChangeMemberVotingStatusForShard with journal batch size 1 07:09:39,517 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 07:09:39,517 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard is initialized 07:09:39,518 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 07:09:39,518 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:39,518 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 07:09:39,518 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 07:09:39,518 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatusForShard from null to Follower 07:09:39,518 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:39,518 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:39,518 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 07:09:39,518 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testChangeMemberVotingStatusForShard is initialized 07:09:39,518 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 07:09:39,518 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:39,519 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 07:09:39,519 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:39,519 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:39,519 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:39,519 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:39,519 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:39,520 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-2099314541 created and ready for shard:member-1-shard-people-config_testChangeMemberVotingStatusForShard 07:09:39,520 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 07:09:39,520 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 07:09:39,520 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatusForShard from null to Follower 07:09:39,523 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 07:09:39,524 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} started 07:09:39,527 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@1170013e} 07:09:39,527 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:39,527 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:39,527 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:39,527 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:39,527 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard#-1063439571] 07:09:39,527 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:39,527 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:39,527 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:39,527 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard on shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard#1188133190] 07:09:39,528 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Shard created : member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, persistent : true 07:09:39,528 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with journal batch size 1 07:09:39,529 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Shard created : member-1-shard-people-oper_testChangeMemberVotingStatusForShard, persistent : true 07:09:39,529 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testChangeMemberVotingStatusForShard with journal batch size 1 07:09:39,529 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard#-1063439571] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,529 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#1823655615 created and ready for shard:member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:39,529 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard/shard-member-1:snapshot-read#1077287321] 07:09:39,529 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#498523799 created and ready for shard:member-1-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:39,529 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard/shard-member-1:snapshot-read#-2034503990] 07:09:39,529 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard#1188133190] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,530 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:39,530 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:39,530 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 07:09:39,530 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:39,531 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 07:09:39,531 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard is initialized 07:09:39,531 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:39,531 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:39,531 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 07:09:39,531 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard is initialized 07:09:39,531 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 07:09:39,531 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 07:09:39,531 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatusForShard from null to Follower 07:09:39,531 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 07:09:39,531 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:39,531 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 07:09:39,531 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 07:09:39,531 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard from null to Follower 07:09:39,532 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:09:39,542 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [7351768785771295519] 07:09:39,542 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 07:09:39,544 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:09:39,544 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 07:09:39,544 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:09:39,544 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:09:39,553 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:39,554 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:39,556 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatusForShard 07:09:39,565 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-1626890803] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,565 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 07:09:39,569 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 07:09:39,569 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:39,569 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testChangeMemberVotingStatusForShard 07:09:39,572 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 07:09:39,573 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@12cccd23} 07:09:39,574 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} started 07:09:39,574 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:39,575 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:39,577 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatusForShard 07:09:39,577 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:39,578 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:39,579 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#-642412051] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,579 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 07:09:39,579 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Shard created : member-2-shard-cars-config_testChangeMemberVotingStatusForShard, persistent : true 07:09:39,580 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testChangeMemberVotingStatusForShard with journal batch size 1 07:09:39,580 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#-316720502] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,580 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard/shard-member-2:snapshot-read#215680997] 07:09:39,580 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#599699082] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,580 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard/shard-member-2:snapshot-read#215680997] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,583 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testChangeMemberVotingStatusForShard 07:09:39,583 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard#218714316] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,583 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Shard created : member-2-shard-people-config_testChangeMemberVotingStatusForShard, persistent : true 07:09:39,584 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-829867282 created and ready for shard:member-2-shard-people-config_testChangeMemberVotingStatusForShard 07:09:39,584 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-829867282] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,584 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard/shard-member-2:snapshot-read#-1620186620] 07:09:39,584 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#599699082 created and ready for shard:member-2-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:39,584 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard/shard-member-2:snapshot-read#-1620186620] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,584 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:39,585 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:39,585 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 07:09:39,585 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard is initialized 07:09:39,585 AM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 07:09:39,585 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:39,585 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 07:09:39,585 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 07:09:39,585 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testChangeMemberVotingStatusForShard from null to Follower 07:09:39,585 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Starting recovery for member-2-shard-people-config_testChangeMemberVotingStatusForShard with journal batch size 1 07:09:39,587 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:39,587 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:39,587 AM [cluster-test-shard-dispatcher-35] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 07:09:39,587 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:39,587 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 07:09:39,587 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testChangeMemberVotingStatusForShard is initialized 07:09:39,588 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 07:09:39,588 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 07:09:39,588 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 07:09:39,588 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testChangeMemberVotingStatusForShard from null to Follower 07:09:39,588 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:39,588 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:39,590 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 07:09:39,590 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@5d3e2a45} 07:09:39,590 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} started 07:09:39,591 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:39,591 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:39,592 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Shard created : member-2-shard-cars-oper_testChangeMemberVotingStatusForShard, persistent : true 07:09:39,593 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with journal batch size 1 07:09:39,593 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:39,593 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Shard created : member-2-shard-people-oper_testChangeMemberVotingStatusForShard, persistent : true 07:09:39,593 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testChangeMemberVotingStatusForShard with journal batch size 1 07:09:39,594 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#1142399397 created and ready for shard:member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:39,594 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard/shard-member-2:snapshot-read#829098367] 07:09:39,594 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#1485301805 created and ready for shard:member-2-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:39,594 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard/shard-member-2:snapshot-read#-1556391799] 07:09:39,594 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard#1137468165] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,594 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard#1543703260] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,596 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:39,596 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:39,596 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 07:09:39,596 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:39,596 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 07:09:39,596 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard is initialized 07:09:39,596 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 07:09:39,596 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 07:09:39,596 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testChangeMemberVotingStatusForShard from null to Follower 07:09:39,596 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:39,596 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:39,596 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 07:09:39,596 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard is initialized 07:09:39,596 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 07:09:39,596 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:39,597 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 07:09:39,597 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 07:09:39,597 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard from null to Follower 07:09:39,599 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 07:09:39,604 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 07:09:39,605 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:39,605 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:39,605 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:39,605 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:39,605 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:39,605 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:39,605 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:39,605 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:39,605 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:39,605 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:39,605 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 07:09:39,605 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 07:09:39,605 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:39,605 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 07:09:39,609 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:09:39,620 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [7800168526435547939] 07:09:39,620 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 07:09:39,622 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:09:39,622 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 07:09:39,623 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 07:09:39,623 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 07:09:39,644 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:39,645 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:39,647 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatusForShard 07:09:39,652 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-284454121] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,653 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 07:09:39,655 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 07:09:39,655 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:39,655 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testChangeMemberVotingStatusForShard 07:09:39,657 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 07:09:39,660 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@62242e33} 07:09:39,660 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} started 07:09:39,661 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:39,662 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 07:09:39,663 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:39,663 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:39,663 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testChangeMemberVotingStatusForShard 07:09:39,664 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-3-shard-cars-config_testChangeMemberVotingStatusForShard, persistent : true 07:09:39,665 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testChangeMemberVotingStatusForShard with journal batch size 1 07:09:39,665 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-3-shard-people-config_testChangeMemberVotingStatusForShard, persistent : true 07:09:39,665 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#1187264357 created and ready for shard:member-3-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:39,665 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard/shard-member-3:snapshot-read#-268799865] 07:09:39,665 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-3-shard-people-config_testChangeMemberVotingStatusForShard with journal batch size 1 07:09:39,665 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatusForShard 07:09:39,666 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 07:09:39,666 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#169861969] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,666 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard#930572542] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,666 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#1187264357] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,666 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard/shard-member-3:snapshot-read#-268799865] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,666 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#734117225] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,666 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard/shard-member-3:snapshot-read#-672641629] 07:09:39,666 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard/shard-member-3:snapshot-read#-672641629] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,667 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-873327556 created and ready for shard:member-3-shard-people-config_testChangeMemberVotingStatusForShard 07:09:39,667 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-873327556] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,667 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 07:09:39,667 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:39,668 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:39,669 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 07:09:39,671 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 07:09:39,671 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:39,671 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:39,671 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:39,672 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 07:09:39,672 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:39,672 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:39,673 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:39,673 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:39,673 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 07:09:39,673 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:39,673 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 07:09:39,673 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:39,675 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 07:09:39,675 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 07:09:39,675 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 07:09:39,675 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard is initialized 07:09:39,675 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 07:09:39,675 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testChangeMemberVotingStatusForShard from null to Follower 07:09:39,680 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:39,680 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:39,680 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 07:09:39,680 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testChangeMemberVotingStatusForShard is initialized 07:09:39,680 AM [cluster-test-shard-dispatcher-35] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 07:09:39,680 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:39,680 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 07:09:39,680 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 07:09:39,680 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 07:09:39,680 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testChangeMemberVotingStatusForShard from null to Follower 07:09:39,685 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 07:09:39,686 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f971c71ed70@361a8033} 07:09:39,686 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} started 07:09:39,686 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 07:09:39,686 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:39,686 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 07:09:39,687 AM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:39,687 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:39,687 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard#504665865] 07:09:39,687 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:39,687 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-3-shard-cars-oper_testChangeMemberVotingStatusForShard, persistent : true 07:09:39,687 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Shard created : member-3-shard-people-oper_testChangeMemberVotingStatusForShard, persistent : true 07:09:39,687 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with journal batch size 1 07:09:39,688 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#802849556 created and ready for shard:member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:39,688 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard/shard-member-3:snapshot-read#-478977540] 07:09:39,688 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard#504665865] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,688 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#-421089350 created and ready for shard:member-3-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:39,688 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard/shard-member-3:snapshot-read#-958821931] 07:09:39,688 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testChangeMemberVotingStatusForShard with journal batch size 1 07:09:39,688 AM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:39,688 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:39,688 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard on shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard#460087318] 07:09:39,688 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard#460087318] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 07:09:39,688 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:39,688 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 07:09:39,688 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:39,688 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:09:39,689 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 07:09:39,689 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard is initialized 07:09:39,689 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 07:09:39,689 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:39,689 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 07:09:39,689 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard is initialized 07:09:39,689 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 07:09:39,689 AM [cluster-test-shard-dispatcher-35] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 07:09:39,689 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:39,689 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:39,689 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 07:09:39,689 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 07:09:39,689 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:39,689 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testChangeMemberVotingStatusForShard from null to Follower 07:09:39,689 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 07:09:39,689 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard from null to Follower 07:09:39,857 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 07:09:39,857 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Received ElectionTimeout - switching to Candidate 07:09:39,857 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-oper_testChangeMemberVotingStatusForShard, member-2-shard-cars-oper_testChangeMemberVotingStatusForShard] 07:09:39,857 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 07:09:39,857 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:09:39,858 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 07:09:39,858 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:39,858 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 07:09:39,859 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard from Follower to Candidate 07:09:39,867 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 07:09:39,867 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Received ElectionTimeout - switching to Candidate 07:09:39,867 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Election: Candidate has following voting peers: [member-3-shard-people-config_testChangeMemberVotingStatusForShard, member-2-shard-people-config_testChangeMemberVotingStatusForShard] 07:09:39,868 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 07:09:39,868 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 07:09:39,868 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Received ElectionTimeout - switching to Candidate 07:09:39,868 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:09:39,868 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-oper_testChangeMemberVotingStatusForShard, member-3-shard-people-oper_testChangeMemberVotingStatusForShard] 07:09:39,868 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 07:09:39,868 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:39,868 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 07:09:39,868 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:09:39,868 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 07:09:39,868 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 07:09:39,868 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:39,868 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 07:09:39,868 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatusForShard from Follower to Candidate 07:09:39,868 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatusForShard from Follower to Candidate 07:09:39,876 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 07:09:39,876 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Received ElectionTimeout - switching to Candidate 07:09:39,876 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testChangeMemberVotingStatusForShard, member-3-shard-cars-config_testChangeMemberVotingStatusForShard] 07:09:39,876 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 07:09:39,876 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:09:39,876 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 07:09:39,876 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 07:09:39,876 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 07:09:39,877 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatusForShard from Follower to Candidate 07:09:40,186 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 07:09:40,186 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 07:09:40,217 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 07:09:40,217 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 07:09:40,257 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 07:09:40,257 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 07:09:40,266 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 07:09:40,266 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 07:09:40,498 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 07:09:40,498 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 07:09:40,498 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:40,498 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:40,498 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:40,498 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:40,498 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:40,498 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:40,498 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:40,498 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:40,498 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:09:40,498 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:40,498 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 07:09:40,498 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:40,498 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 07:09:40,498 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:40,499 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:40,499 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:09:40,499 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:40,499 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:40,499 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:40,499 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 07:09:40,499 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 07:09:40,499 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:40,499 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:40,499 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:40,499 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:40,499 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:40,499 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 07:09:40,499 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 07:09:40,505 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:40,505 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:40,505 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:40,505 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:40,505 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:40,505 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:09:40,505 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:40,505 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:40,505 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:40,505 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:40,505 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:40,505 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:40,505 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:40,505 AM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:40,505 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:40,505 AM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 07:09:40,505 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 07:09:40,505 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:09:40,505 AM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:40,505 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:40,505 AM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 07:09:40,505 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 07:09:40,505 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 07:09:40,507 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:40,571 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:40,571 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 07:09:40,571 AM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:40,571 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:40,571 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:40,571 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:40,571 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:40,571 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:40,571 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:09:40,571 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:40,571 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:40,571 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:40,571 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:40,571 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:40,572 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:40,572 AM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 07:09:40,572 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 07:09:40,572 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 07:09:40,572 AM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:40,572 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:40,572 AM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 07:09:40,572 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:40,572 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 07:09:40,572 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 07:09:40,606 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 07:09:40,606 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 07:09:40,606 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1} to peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:40,607 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1} to peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:40,608 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:40,608 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:40,608 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Cluster state: [] 07:09:40,608 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Cluster state: [] 07:09:40,608 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:09:40,608 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:09:40,608 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:09:40,608 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:09:40,608 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:09:40,608 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:09:40,608 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:09:40,608 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:09:40,609 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): RaftRPC message received RequestVoteReply{term=3, voteGranted=true}, my term is 3 07:09:40,609 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): handleRequestVoteReply: RequestVoteReply{term=3, voteGranted=true}, current voteCount: 1 07:09:40,609 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Leader): Election: Leader has following peers: [member-3-shard-cars-oper_testChangeMemberVotingStatusForShard, member-2-shard-cars-oper_testChangeMemberVotingStatusForShard] 07:09:40,609 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:40,609 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:40,609 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 07:09:40,609 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 07:09:40,609 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 07:09:40,609 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 07:09:40,609 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 07:09:40,609 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard from Candidate to Leader 07:09:40,610 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 07:09:40,610 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard status sync done false 07:09:40,610 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard status sync done false 07:09:40,610 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 07:09:40,611 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard updated: matchIndex: -1, nextIndex: 0 07:09:40,611 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:40,611 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard updated: matchIndex: -1, nextIndex: 0 07:09:40,611 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:40,616 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 07:09:40,616 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 07:09:40,616 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 07:09:40,616 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1} to peer member-3-shard-people-config_testChangeMemberVotingStatusForShard 07:09:40,616 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 07:09:40,616 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1} to peer member-2-shard-people-config_testChangeMemberVotingStatusForShard 07:09:40,616 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1} to peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:40,617 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1} to peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:40,618 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:40,618 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Cluster state: [] 07:09:40,618 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:09:40,618 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:09:40,618 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:09:40,618 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:09:40,618 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:40,618 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Cluster state: [] 07:09:40,618 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:09:40,618 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:09:40,618 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:09:40,618 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:09:40,618 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:40,618 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Cluster state: [] 07:09:40,618 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:09:40,618 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:09:40,618 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:09:40,618 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:09:40,618 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:40,618 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Cluster state: [] 07:09:40,618 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:09:40,618 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:09:40,618 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:09:40,618 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:09:40,619 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): RaftRPC message received RequestVoteReply{term=3, voteGranted=true}, my term is 3 07:09:40,619 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): handleRequestVoteReply: RequestVoteReply{term=3, voteGranted=true}, current voteCount: 1 07:09:40,619 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Election: Leader has following peers: [member-3-shard-people-config_testChangeMemberVotingStatusForShard, member-2-shard-people-config_testChangeMemberVotingStatusForShard] 07:09:40,619 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:40,619 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:40,619 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 07:09:40,620 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 07:09:40,620 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 07:09:40,620 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 07:09:40,620 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): RaftRPC message received RequestVoteReply{term=3, voteGranted=true}, my term is 3 07:09:40,620 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): handleRequestVoteReply: RequestVoteReply{term=3, voteGranted=true}, current voteCount: 1 07:09:40,620 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 07:09:40,620 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatusForShard from Candidate to Leader 07:09:40,620 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testChangeMemberVotingStatusForShard, member-3-shard-cars-config_testChangeMemberVotingStatusForShard] 07:09:40,620 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:40,620 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:40,620 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 07:09:40,620 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 07:09:40,620 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 07:09:40,620 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 07:09:40,620 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 07:09:40,620 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatusForShard from Candidate to Leader 07:09:40,620 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testChangeMemberVotingStatusForShard is ready 07:09:40,620 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} is now ready 07:09:40,620 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 07:09:40,622 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 07:09:40,622 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testChangeMemberVotingStatusForShard status sync done false 07:09:40,622 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 07:09:40,622 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testChangeMemberVotingStatusForShard is ready 07:09:40,622 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatusForShard status sync done false 07:09:40,624 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testChangeMemberVotingStatusForShard status sync done false 07:09:40,624 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatusForShard status sync done false 07:09:40,624 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 07:09:40,624 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 07:09:40,624 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testChangeMemberVotingStatusForShard updated: matchIndex: -1, nextIndex: 0 07:09:40,624 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testChangeMemberVotingStatusForShard updated: matchIndex: -1, nextIndex: 0 07:09:40,624 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testChangeMemberVotingStatusForShard is ready 07:09:40,624 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:40,624 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:40,624 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} is now ready 07:09:40,624 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#1840577073] 07:09:40,625 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:09:40,625 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-59003840] 07:09:40,625 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-59003840] 07:09:40,626 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testChangeMemberVotingStatusForShard updated: matchIndex: -1, nextIndex: 0 07:09:40,626 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:40,626 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testChangeMemberVotingStatusForShard updated: matchIndex: -1, nextIndex: 0 07:09:40,626 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-1626890803] 07:09:40,626 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:40,626 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 07:09:40,626 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#-316720502] 07:09:40,626 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#-316720502] 07:09:40,626 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-284454121] 07:09:40,626 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 07:09:40,626 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#169861969] 07:09:40,626 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#169861969] 07:09:40,626 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 07:09:40,626 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 07:09:40,626 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1} to peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:40,627 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1} to peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:40,626 AM [main] [INFO] ClusterAdminRpcService - Change member voting states for shard cars: {member-2=false, member-3=false} 07:09:40,627 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-2=false, member-3=false}] 07:09:40,627 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 07:09:40,628 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatusForShard=false, member-3-shard-cars-config_testChangeMemberVotingStatusForShard=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:40,628 AM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatusForShard=false, member-3-shard-cars-config_testChangeMemberVotingStatusForShard=false}, serversVisited=[]], state: Idle 07:09:40,628 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: getLeaderAddress leaderId = member-1-shard-cars-config_testChangeMemberVotingStatusForShard peerAddress = pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:40,628 AM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Not leader - forwarding to leader ActorSelection[Anchor(pekko://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard)] 07:09:40,631 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatusForShard=false, member-3-shard-cars-config_testChangeMemberVotingStatusForShard=false}, serversVisited=[]], state: Idle 07:09:40,631 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 07:09:40,631 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard, votingState=NON_VOTING]] 07:09:40,631 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]] 07:09:40,631 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 07:09:40,631 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 07:09:40,631 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=58003514-14ca-44df-b60d-9dcf4f2206a3}, logIndex: 0, isSendImmediate: true 07:09:40,631 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Setting last applied to 0 07:09:40,631 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Moving last applied index from -1 to 0 07:09:40,631 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]] 07:09:40,631 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:40,631 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:40,631 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard: AppendEntries{term=3, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]]]} 07:09:40,631 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:40,631 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:40,631 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard: AppendEntries{term=3, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]]]} 07:09:40,631 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatusForShard=false, member-3-shard-cars-config_testChangeMemberVotingStatusForShard=false}, serversVisited=[]] 07:09:40,631 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]] has been successfully replicated to a majority of followers 07:09:40,631 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Persisting ApplyJournalEntries with index=0 07:09:40,632 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:40,632 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Cluster state: [] 07:09:40,632 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:09:40,632 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:09:40,632 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:09:40,632 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:09:40,632 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 07:09:40,633 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 07:09:40,633 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Cluster state: [] 07:09:40,633 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 07:09:40,633 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 07:09:40,633 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): In requestVote: RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 07:09:40,633 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 07:09:40,634 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): RaftRPC message received RequestVoteReply{term=3, voteGranted=true}, my term is 3 07:09:40,634 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): handleRequestVoteReply: RequestVoteReply{term=3, voteGranted=true}, current voteCount: 1 07:09:40,634 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): Election: Leader has following peers: [member-2-shard-people-oper_testChangeMemberVotingStatusForShard, member-3-shard-people-oper_testChangeMemberVotingStatusForShard] 07:09:40,634 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:40,634 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:40,634 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 07:09:40,634 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 07:09:40,634 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 07:09:40,634 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 07:09:40,634 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 07:09:40,634 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatusForShard from Candidate to Leader 07:09:40,634 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testChangeMemberVotingStatusForShard is ready 07:09:40,635 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]]]} 07:09:40,635 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Number of entries to be appended = 1 07:09:40,635 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 07:09:40,635 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]] 07:09:40,635 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 07:09:40,635 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard, votingState=NON_VOTING]] 07:09:40,635 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testChangeMemberVotingStatusForShard} from local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:40,635 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Log size is now 1 07:09:40,635 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 07:09:40,635 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Commit index set to 0 07:09:40,635 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:40,635 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 07:09:40,635 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Setting last applied to 0 07:09:40,635 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Moving last applied index from -1 to 0 07:09:40,635 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]] 07:09:40,635 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ClusterAdminRpcService - Successfully changed member voting states for shard cars 07:09:40,635 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testChangeMemberVotingStatusForShard status sync done false 07:09:40,636 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 07:09:40,636 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]]]} 07:09:40,636 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Number of entries to be appended = 1 07:09:40,636 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 07:09:40,636 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]] 07:09:40,636 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 07:09:40,636 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard, votingState=VOTING]] 07:09:40,636 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Log size is now 1 07:09:40,636 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Commit index set to 0 07:09:40,636 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 07:09:40,636 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testChangeMemberVotingStatusForShard status sync done false 07:09:40,636 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 07:09:40,636 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Setting last applied to 0 07:09:40,636 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Moving last applied index from -1 to 0 07:09:40,636 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]] 07:09:40,636 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 07:09:40,636 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testChangeMemberVotingStatusForShard is ready 07:09:40,636 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 07:09:40,637 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Persisting ApplyJournalEntries with index=0 07:09:40,637 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatusForShard status sync done true 07:09:40,637 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 07:09:40,637 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testChangeMemberVotingStatusForShard is ready 07:09:40,637 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#1840577073] 07:09:40,637 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 07:09:40,637 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 07:09:40,637 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Persisting ApplyJournalEntries with index=0 07:09:40,637 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-59003840] 07:09:40,637 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatusForShard status sync done true 07:09:40,637 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-59003840] 07:09:40,637 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testChangeMemberVotingStatusForShard updated: matchIndex: -1, nextIndex: 0 07:09:40,637 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-1626890803] 07:09:40,637 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:40,637 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testChangeMemberVotingStatusForShard updated: matchIndex: 0, nextIndex: 1 07:09:40,637 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:40,637 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 07:09:40,638 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#-316720502] 07:09:40,638 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testChangeMemberVotingStatusForShard updated: matchIndex: 0, nextIndex: 1 07:09:40,638 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#-316720502] 07:09:40,638 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:40,638 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testChangeMemberVotingStatusForShard updated: matchIndex: -1, nextIndex: 0 07:09:40,638 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:40,638 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-284454121] 07:09:40,638 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 07:09:40,638 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#169861969] 07:09:40,638 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#169861969] 07:09:40,638 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 07:09:40,638 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:40,638 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testChangeMemberVotingStatusForShard 07:09:40,638 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onShutDown 07:09:40,638 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 07:09:40,638 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: onShutDown 07:09:40,638 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:40,639 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Stopping Shard member-3-shard-people-config_testChangeMemberVotingStatusForShard 07:09:40,639 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 07:09:40,639 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 07:09:40,639 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 07:09:40,639 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:40,639 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:40,639 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: onShutDown 07:09:40,639 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 07:09:40,639 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: onShutDown 07:09:40,639 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:40,640 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:40,640 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 07:09:40,640 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 07:09:40,641 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:09:40,641 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:09:40,643 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:40,643 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:40,644 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:40,644 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:40,655 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 07:09:40,655 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:40,655 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testChangeMemberVotingStatusForShard 07:09:40,655 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 07:09:40,655 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: onShutDown 07:09:40,655 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: onShutDown 07:09:40,656 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:40,656 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-2-shard-people-config_testChangeMemberVotingStatusForShard 07:09:40,657 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 07:09:40,657 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 07:09:40,657 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 07:09:40,657 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:40,658 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:40,658 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 07:09:40,658 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: onShutDown 07:09:40,658 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: onShutDown 07:09:40,658 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:40,658 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:40,658 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 07:09:40,658 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 07:09:40,659 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:09:40,659 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:09:41,518 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 07:09:41,657 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [7800168526435547939] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 07:09:41,677 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [7351768785771295519] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 07:09:41,681 AM [cluster-test-pekko.actor.default-dispatcher-32] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:41,681 AM [cluster-test-pekko.actor.default-dispatcher-32] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:09:41,683 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [01/31/2025 07:09:41.686] [cluster-test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:09:41,698 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 07:09:41,699 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:41,699 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testChangeMemberVotingStatusForShard 07:09:41,699 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 07:09:41,699 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onShutDown 07:09:41,699 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Initiating leader transfer 07:09:41,699 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onShutDown 07:09:41,700 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Initiating leader transfer 07:09:41,700 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6f9e5c34 07:09:41,700 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6f9e5c34 07:09:41,700 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 07:09:41,700 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Attempting to transfer leadership 07:09:41,700 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:41,700 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 07:09:41,700 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@28d24951 07:09:41,700 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@28d24951 07:09:41,700 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 07:09:41,700 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Attempting to transfer leadership 07:09:41,700 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:41,700 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:41,700 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, leaderId=null, leaderPayloadVersion=13] 07:09:41,700 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, leaderId=null, leaderPayloadVersion=13] 07:09:42,216 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: leader transfer aborted 07:09:42,216 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatusForShard: leader transfer aborted 07:09:42,216 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 516.9 ms 07:09:42,216 AM [cluster-test-shard-dispatcher-20] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 516.8 ms 07:09:42,217 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: In unpauseLeader 07:09:42,217 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: In unpauseLeader 07:09:42,217 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: leader transfer failed - sending PoisonPill 07:09:42,217 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: leader transfer failed - sending PoisonPill 07:09:42,217 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Leadership transfer expired 07:09:42,217 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Leadership transfer expired 07:09:42,217 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-people-config_testChangeMemberVotingStatusForShard 07:09:42,217 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard 07:09:42,218 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 07:09:42,218 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 07:09:42,218 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 07:09:42,219 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:42,219 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:42,219 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 07:09:42,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onShutDown 07:09:42,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Initiating leader transfer 07:09:42,219 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onShutDown 07:09:42,219 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Initiating leader transfer 07:09:42,219 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, leaderId=null, leaderPayloadVersion=13] 07:09:42,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6d88f068 07:09:42,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6d88f068 07:09:42,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 07:09:42,219 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, leaderId=null, leaderPayloadVersion=13] 07:09:42,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Leader): Attempting to transfer leadership 07:09:42,219 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@371c8193 07:09:42,219 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@371c8193 07:09:42,219 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 07:09:42,219 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): Attempting to transfer leadership 07:09:42,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:42,219 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:42,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:42,219 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 07:09:42,527 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: leader transfer aborted 07:09:42,527 AM [cluster-test-shard-dispatcher-20] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 307.7 ms 07:09:42,527 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: In unpauseLeader 07:09:42,527 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: leader transfer failed - sending PoisonPill 07:09:42,527 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Leader): Leadership transfer expired 07:09:42,528 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 07:09:42,547 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: leader transfer aborted 07:09:42,547 AM [cluster-test-shard-dispatcher-20] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 328.0 ms 07:09:42,547 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: In unpauseLeader 07:09:42,547 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: leader transfer failed - sending PoisonPill 07:09:42,547 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): Leadership transfer expired 07:09:42,547 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard 07:09:42,548 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 07:09:42,548 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 07:09:42,549 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:09:42,549 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:09:42,552 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [01/31/2025 07:09:42.554] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config