06:44:28,643 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 06:44:29,205 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [8678706475887985158] 06:44:29,226 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 06:44:29,381 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 06:44:29,381 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 06:44:29,466 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:44:29,466 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:44:29,769 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:29,773 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:29,774 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 06:44:29,783 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 06:44:29,793 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 06:44:30,637 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testBackupDatastore 06:44:30,775 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-1699198027] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:30,784 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 06:44:30,785 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 06:44:30,823 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 06:44:30,827 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testBackupDatastore 06:44:30,837 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testBackupDatastore: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testBackupDatastore, generation=0} 06:44:30,884 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testBackupDatastore 06:44:30,906 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:44:30,906 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore 06:44:30,906 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore 06:44:30,914 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@b0895af} 06:44:30,933 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testBackupDatastore, generation=0} started 06:44:30,939 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:44:30,939 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testBackupDatastore 06:44:30,942 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore 06:44:30,942 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore to pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore 06:44:30,944 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testBackupDatastore on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore on shard member-1-shard-cars-config_testBackupDatastore whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#-1276592382] 06:44:30,944 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testBackupDatastore 06:44:30,956 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore 06:44:30,956 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore to pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore 06:44:30,956 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testBackupDatastore on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore on shard member-1-shard-people-config_testBackupDatastore whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore#-723459419] 06:44:30,958 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#-1276592382] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:30,958 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore#-723459419] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:30,989 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-1699198027] 06:44:30,990 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:44:30,990 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 06:44:30,999 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testBackupDatastore: Shard created, persistent : true 06:44:30,999 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testBackupDatastore: Shard created, persistent : true 06:44:31,41 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-1699198027] 06:44:31,42 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:44:31,42 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 06:44:31,92 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-1699198027] 06:44:31,93 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:44:31,93 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 06:44:31,112 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore-notifier#506335168] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:31,113 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore-notifier#1207499906] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:31,114 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore-notifier#1207499906 created and ready for shard:member-1-shard-cars-config_testBackupDatastore 06:44:31,116 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore-notifier#506335168 created and ready for shard:member-1-shard-people-config_testBackupDatastore 06:44:31,121 PM [cluster-test-shard-dispatcher-16] [INFO] RaftActor - member-1-shard-cars-config_testBackupDatastore: Starting recovery with journal batch size 1 06:44:31,123 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-people-config_testBackupDatastore: Starting recovery with journal batch size 1 06:44:31,144 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-1699198027] 06:44:31,145 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:44:31,145 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 06:44:31,195 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-1699198027] 06:44:31,196 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:44:31,196 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 06:44:31,246 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-1699198027] 06:44:31,247 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:44:31,247 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 06:44:31,298 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-1699198027] 06:44:31,298 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:44:31,298 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 06:44:31,313 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-config_testBackupDatastore: journal open: applyTo=0 06:44:31,313 PM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testBackupDatastore: journal open: applyTo=0 06:44:31,340 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:31,341 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:31,349 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-1699198027] 06:44:31,349 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:44:31,350 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 06:44:31,352 PM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-cars-config_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:44:31,354 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-config_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:44:31,371 PM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-cars-config_testBackupDatastore: Local TermInfo store seeded with TermInfo{term=0} 06:44:31,373 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-config_testBackupDatastore: Local TermInfo store seeded with TermInfo{term=0} 06:44:31,377 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 06:44:31,378 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testBackupDatastore is initialized 06:44:31,379 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 06:44:31,379 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testBackupDatastore is initialized 06:44:31,382 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 06:44:31,382 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:31,383 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , received role change from null to Follower 06:44:31,384 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 06:44:31,384 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:31,384 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , received role change from null to Follower 06:44:31,400 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-1699198027] 06:44:31,400 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:44:31,401 PM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#-1276592382] 06:44:31,403 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-config_testBackupDatastore (Candidate): Starting new election term 1 06:44:31,403 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-config_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:44:31,403 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 06:44:31,403 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:31,403 PM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-cars-config_testBackupDatastore (Candidate): Starting new election term 1 06:44:31,404 PM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-cars-config_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:44:31,404 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 06:44:31,404 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:31,407 PM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-cars-config_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 06:44:31,407 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-config_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 06:44:31,407 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 06:44:31,407 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 06:44:31,407 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , received role change from Follower to Candidate 06:44:31,407 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , received role change from Follower to Candidate 06:44:31,411 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: became leader with frontend state for [] 06:44:31,411 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: became leader with frontend state for [] 06:44:31,411 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , received role change from Candidate to Leader 06:44:31,411 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , received role change from Candidate to Leader 06:44:31,417 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , registered listener pekko://cluster-test/user/shardmanager-config_testBackupDatastore 06:44:31,418 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7ec548fd 06:44:31,418 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , registered listener pekko://cluster-test/user/shardmanager-config_testBackupDatastore 06:44:31,418 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testBackupDatastore from Candidate to Leader 06:44:31,418 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1100ea3d 06:44:31,419 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testBackupDatastore from Candidate to Leader 06:44:31,419 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testBackupDatastore is ready 06:44:31,468 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-1699198027] 06:44:31,469 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 06:44:31,469 PM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore#-723459419] 06:44:31,469 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:31,475 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:31,480 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testBackupDatastore 06:44:31,483 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore#1576570698] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:31,483 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 06:44:31,484 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 06:44:31,486 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 06:44:31,486 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testBackupDatastore 06:44:31,486 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testBackupDatastore 06:44:31,487 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:44:31,487 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore 06:44:31,487 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore 06:44:31,488 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testBackupDatastore: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testBackupDatastore, generation=0} 06:44:31,489 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@3117462f} 06:44:31,490 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testBackupDatastore, generation=0} started 06:44:31,490 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:44:31,490 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testBackupDatastore 06:44:31,491 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore 06:44:31,491 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore to pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore 06:44:31,491 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testBackupDatastore on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore on shard member-1-shard-cars-oper_testBackupDatastore whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore#-96158723] 06:44:31,492 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testBackupDatastore 06:44:31,492 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testBackupDatastore: Shard created, persistent : true 06:44:31,492 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore#-96158723] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:31,492 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore 06:44:31,493 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore to pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore 06:44:31,493 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testBackupDatastore on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore on shard member-1-shard-people-oper_testBackupDatastore whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore#-192309811] 06:44:31,493 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-oper_testBackupDatastore: Starting recovery with journal batch size 1 06:44:31,493 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-oper_testBackupDatastore: Shard created, persistent : true 06:44:31,493 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore-notifier#1495603881 created and ready for shard:member-1-shard-cars-oper_testBackupDatastore 06:44:31,493 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore-notifier#1495603881] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:31,494 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore#-192309811] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:31,494 PM [cluster-test-shard-dispatcher-16] [INFO] RaftActor - member-1-shard-people-oper_testBackupDatastore: Starting recovery with journal batch size 1 06:44:31,495 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testBackupDatastore: journal open: applyTo=0 06:44:31,495 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore-notifier#167120815 created and ready for shard:member-1-shard-people-oper_testBackupDatastore 06:44:31,496 PM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testBackupDatastore: journal open: applyTo=0 06:44:31,498 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:31,498 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore-notifier#167120815] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:31,498 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:31,498 PM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-cars-oper_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:44:31,498 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-oper_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:44:31,501 PM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-cars-oper_testBackupDatastore: Local TermInfo store seeded with TermInfo{term=0} 06:44:31,502 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 06:44:31,502 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 06:44:31,502 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testBackupDatastore is initialized 06:44:31,502 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:31,502 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-oper_testBackupDatastore: Local TermInfo store seeded with TermInfo{term=0} 06:44:31,502 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , received role change from null to Follower 06:44:31,503 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 06:44:31,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 06:44:31,503 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testBackupDatastore is initialized 06:44:31,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:31,503 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , received role change from null to Follower 06:44:31,506 PM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-cars-oper_testBackupDatastore (Candidate): Starting new election term 1 06:44:31,506 PM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:44:31,506 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 06:44:31,506 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:31,506 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , received role change from Follower to Candidate 06:44:31,506 PM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 06:44:31,506 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-oper_testBackupDatastore (Candidate): Starting new election term 1 06:44:31,506 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 06:44:31,506 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-oper_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:44:31,506 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: became leader with frontend state for [] 06:44:31,506 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 06:44:31,507 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:31,507 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , received role change from Candidate to Leader 06:44:31,507 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , received role change from Follower to Candidate 06:44:31,507 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-oper_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 06:44:31,507 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , registered listener pekko://cluster-test/user/shardmanager-oper_testBackupDatastore 06:44:31,507 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 06:44:31,507 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: became leader with frontend state for [] 06:44:31,507 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@55832c37 06:44:31,507 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , received role change from Candidate to Leader 06:44:31,507 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testBackupDatastore from Candidate to Leader 06:44:31,507 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , registered listener pekko://cluster-test/user/shardmanager-oper_testBackupDatastore 06:44:31,508 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@73fb6a37 06:44:31,508 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testBackupDatastore from Candidate to Leader 06:44:31,508 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testBackupDatastore is ready 06:44:31,540 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore#1576570698] 06:44:31,541 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 06:44:31,541 PM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore#-96158723] 06:44:31,592 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore#1576570698] 06:44:31,592 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 06:44:31,593 PM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore#-192309811] 06:44:31,613 PM [main] [DEBUG] ClusterAdminRpcService - backupDatastore: BackupDatastoreInput{filePath=target/testBackupDatastore} 06:44:31,614 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: onGetSnapshot 06:44:31,615 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onGetSnapshot 06:44:31,619 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Expecting 2 shard snapshot replies 06:44:31,619 PM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Expecting 2 shard snapshot replies 06:44:31,638 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Received GetSnapshotReply[id=member-1-shard-people-config_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@33c12137, electionTerm=1, electionVotedFor=member-1-shard-people-config_testBackupDatastore, ServerConfigPayload=null]] 06:44:31,650 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Received GetSnapshotReply[id=member-1-shard-cars-config_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@31aa7b18, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testBackupDatastore, ServerConfigPayload=null]] 06:44:31,650 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: All shard snapshots received 06:44:31,638 PM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply[id=member-1-shard-people-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@1030c947, electionTerm=1, electionVotedFor=member-1-shard-people-oper_testBackupDatastore, ServerConfigPayload=null]] 06:44:31,651 PM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply[id=member-1-shard-cars-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@2bb4ed25, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testBackupDatastore, ServerConfigPayload=null]] 06:44:31,651 PM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: All shard snapshots received 06:44:31,704 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ClusterAdminRpcService - Successfully backed up datastore to file target/testBackupDatastore 06:44:31,725 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-1699198027] 06:44:31,725 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:44:31,725 PM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#-1276592382] 06:44:31,763 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testBackupDatastore: Stopping Shard 06:44:31,766 PM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testBackupDatastore: journal closed 06:44:31,772 PM [main] [DEBUG] ClusterAdminRpcService - backupDatastore: BackupDatastoreInput{filePath=target/testBackupDatastore} 06:44:31,773 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: onGetSnapshot 06:44:31,773 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onGetSnapshot 06:44:31,774 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Expecting 2 shard snapshot replies 06:44:31,774 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Expecting 2 shard snapshot replies 06:44:31,774 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply[id=member-1-shard-cars-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@2eabf0c5, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testBackupDatastore, ServerConfigPayload=null]] 06:44:31,775 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply[id=member-1-shard-people-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@5167a1cb, electionTerm=1, electionVotedFor=member-1-shard-people-oper_testBackupDatastore, ServerConfigPayload=null]] 06:44:31,775 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: All shard snapshots received 06:44:31,776 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Received GetSnapshotReply[id=member-1-shard-people-config_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@20e34151, electionTerm=1, electionVotedFor=member-1-shard-people-config_testBackupDatastore, ServerConfigPayload=null]] 06:44:31,997 PM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Timed out after 200 ms while waiting for snapshot replies from 2 shard(s). 1 shard(s) [cars] did not respond 06:44:31,999 PM [cluster-test-pekko.actor.default-dispatcher-14] [ERROR] ClusterAdminRpcService - Failed to back up datastore to file target/testBackupDatastore java.util.concurrent.TimeoutException: Timed out after 200 ms while waiting for snapshot replies from 2 shard(s). 1 shard(s) [cars] did not respond. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManagerGetSnapshotReplyActor.onReceive(ShardManagerGetSnapshotReplyActor.java:66) at org.apache.pekko.actor.UntypedAbstractActor$$anonfun$receive$1.applyOrElse(AbstractActor.scala:341) at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.apache.pekko.actor.AbstractActor.aroundReceive(AbstractActor.scala:229) at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 06:44:32,12 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testBackupDatastore, generation=0} 06:44:32,18 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testBackupDatastore 06:44:32,20 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testBackupDatastore 06:44:32,20 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 06:44:32,22 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@14878957 06:44:32,22 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-people-config_testBackupDatastore: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@14878957 06:44:32,22 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testBackupDatastore: Stopping Shard 06:44:32,23 PM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-people-config_testBackupDatastore: journal closed 06:44:32,25 PM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 06:44:32,26 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 06:44:32,31 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testBackupDatastore, generation=0} 06:44:32,32 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testBackupDatastore 06:44:32,32 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testBackupDatastore 06:44:32,32 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@3ff646ef 06:44:32,32 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testBackupDatastore: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@3ff646ef 06:44:32,32 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@429dacf0 06:44:32,32 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-oper_testBackupDatastore: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@429dacf0 06:44:32,33 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 06:44:32,33 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testBackupDatastore: Stopping Shard 06:44:32,33 PM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testBackupDatastore: journal closed 06:44:32,33 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testBackupDatastore: Stopping Shard 06:44:32,34 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testBackupDatastore: journal closed 06:44:32,34 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 06:44:32,34 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 06:44:32,43 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:44:32,44 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:44:32,70 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 06:44:32,134 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 06:44:32,166 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-6870795606207239478] 06:44:32,167 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 06:44:32,171 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 06:44:32,171 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 06:44:32,185 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:44:32,185 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:44:32,198 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 06:44:32,198 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 06:44:32,198 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 06:44:32,199 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:32,203 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:32,208 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddReplicasForAllShards 06:44:32,216 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#598703368] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:32,217 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 06:44:32,217 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 06:44:32,222 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 06:44:32,222 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testAddReplicasForAllShards 06:44:32,223 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testAddReplicasForAllShards 06:44:32,223 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:44:32,223 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 06:44:32,223 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 06:44:32,224 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 06:44:32,226 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddReplicasForAllShards, generation=0} started 06:44:32,226 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@58d6f367} 06:44:32,226 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:44:32,226 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testAddReplicasForAllShards 06:44:32,227 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 06:44:32,227 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 06:44:32,228 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testAddReplicasForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards on shard member-1-shard-cars-config_testAddReplicasForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-1070578412] 06:44:32,228 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testAddReplicasForAllShards 06:44:32,229 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 06:44:32,229 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 06:44:32,229 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testAddReplicasForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards on shard member-1-shard-people-config_testAddReplicasForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#-1673069384] 06:44:32,230 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-1070578412] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:32,230 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Shard created, persistent : true 06:44:32,230 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#-1673069384] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:32,231 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Shard created, persistent : true 06:44:32,232 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards-notifier#650602019 created and ready for shard:member-1-shard-people-config_testAddReplicasForAllShards 06:44:32,232 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards-notifier#650602019] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:32,232 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards-notifier#-1397953680 created and ready for shard:member-1-shard-cars-config_testAddReplicasForAllShards 06:44:32,236 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-people-config_testAddReplicasForAllShards: Starting recovery with journal batch size 1 06:44:32,237 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-config_testAddReplicasForAllShards: journal open: applyTo=0 06:44:32,238 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:32,238 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:44:32,239 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards-notifier#-1397953680] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:32,241 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-1-shard-cars-config_testAddReplicasForAllShards: Starting recovery with journal batch size 1 06:44:32,242 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-config_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 06:44:32,243 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testAddReplicasForAllShards: journal open: applyTo=0 06:44:32,244 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 06:44:32,244 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testAddReplicasForAllShards is initialized 06:44:32,244 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 06:44:32,244 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:32,246 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from null to Follower 06:44:32,246 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:32,246 PM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-cars-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:44:32,249 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-config_testAddReplicasForAllShards (Candidate): Starting new election term 1 06:44:32,249 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-config_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:44:32,249 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 06:44:32,249 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:32,250 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-config_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 06:44:32,250 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 06:44:32,250 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: became leader with frontend state for [] 06:44:32,250 PM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-cars-config_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 06:44:32,250 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 06:44:32,250 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:32,251 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 06:44:32,251 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from Follower to Candidate 06:44:32,251 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from Candidate to Leader 06:44:32,251 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 06:44:32,251 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testAddReplicasForAllShards is initialized 06:44:32,252 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6ab60457 06:44:32,252 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testAddReplicasForAllShards from Candidate to Leader 06:44:32,251 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from null to Follower 06:44:32,253 PM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-cars-config_testAddReplicasForAllShards (Candidate): Starting new election term 1 06:44:32,253 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:44:32,253 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 06:44:32,254 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:32,254 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 06:44:32,254 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 06:44:32,254 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: became leader with frontend state for [] 06:44:32,254 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from Follower to Candidate 06:44:32,254 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from Candidate to Leader 06:44:32,254 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 06:44:32,255 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1ae25044 06:44:32,255 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddReplicasForAllShards from Candidate to Leader 06:44:32,255 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddReplicasForAllShards is ready 06:44:32,276 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#598703368] 06:44:32,277 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:44:32,277 PM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-1070578412] 06:44:32,328 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#598703368] 06:44:32,328 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 06:44:32,328 PM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#-1673069384] 06:44:32,329 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:32,334 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:32,340 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddReplicasForAllShards 06:44:32,343 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 06:44:32,343 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 06:44:32,344 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#477721803] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:32,346 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 06:44:32,346 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testAddReplicasForAllShards 06:44:32,346 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testAddReplicasForAllShards 06:44:32,347 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:44:32,347 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 06:44:32,347 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 06:44:32,352 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 06:44:32,353 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@37e6ac0d} 06:44:32,353 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} started 06:44:32,354 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:44:32,354 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testAddReplicasForAllShards 06:44:32,355 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Shard created, persistent : true 06:44:32,355 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#962045525] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:32,356 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-oper_testAddReplicasForAllShards: Starting recovery with journal batch size 1 06:44:32,358 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testAddReplicasForAllShards: journal open: applyTo=0 06:44:32,358 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards-notifier#208946349 created and ready for shard:member-1-shard-cars-oper_testAddReplicasForAllShards 06:44:32,359 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards-notifier#208946349] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:32,360 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:32,360 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:44:32,361 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 06:44:32,361 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 06:44:32,361 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testAddReplicasForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards on shard member-1-shard-cars-oper_testAddReplicasForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#962045525] 06:44:32,361 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testAddReplicasForAllShards 06:44:32,362 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 06:44:32,362 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 06:44:32,362 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testAddReplicasForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards on shard member-1-shard-people-oper_testAddReplicasForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#49494200] 06:44:32,363 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Shard created, persistent : true 06:44:32,363 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#49494200] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:32,364 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-1-shard-people-oper_testAddReplicasForAllShards: Starting recovery with journal batch size 1 06:44:32,364 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards-notifier#-926818361 created and ready for shard:member-1-shard-people-oper_testAddReplicasForAllShards 06:44:32,364 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards-notifier#-926818361] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:32,365 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-oper_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 06:44:32,365 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 06:44:32,365 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:32,366 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 06:44:32,366 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testAddReplicasForAllShards is initialized 06:44:32,366 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from null to Follower 06:44:32,366 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testAddReplicasForAllShards: journal open: applyTo=0 06:44:32,368 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-oper_testAddReplicasForAllShards (Candidate): Starting new election term 1 06:44:32,368 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:44:32,368 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 06:44:32,368 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:32,368 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 06:44:32,368 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 06:44:32,368 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: became leader with frontend state for [] 06:44:32,368 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from Follower to Candidate 06:44:32,368 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from Candidate to Leader 06:44:32,369 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 06:44:32,369 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@33e2ed48 06:44:32,369 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddReplicasForAllShards from Candidate to Leader 06:44:32,369 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:32,369 PM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-people-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:44:32,372 PM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-people-oper_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 06:44:32,372 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 06:44:32,372 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testAddReplicasForAllShards is initialized 06:44:32,372 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 06:44:32,372 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:32,372 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from null to Follower 06:44:32,375 PM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-people-oper_testAddReplicasForAllShards (Candidate): Starting new election term 1 06:44:32,375 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-people-oper_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:44:32,375 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 06:44:32,375 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:32,375 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from Follower to Candidate 06:44:32,375 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-oper_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 06:44:32,375 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 06:44:32,375 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: became leader with frontend state for [] 06:44:32,376 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from Candidate to Leader 06:44:32,376 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 06:44:32,376 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4aa254b8 06:44:32,376 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testAddReplicasForAllShards from Candidate to Leader 06:44:32,376 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddReplicasForAllShards is ready 06:44:32,404 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#477721803] 06:44:32,405 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 06:44:32,405 PM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#962045525] 06:44:32,456 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#477721803] 06:44:32,456 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 06:44:32,456 PM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#49494200] 06:44:32,462 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@2663c651] 06:44:32,468 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1 doCreateShard: shardId: member-1-shard-pets-config_testAddReplicasForAllShards, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: true 06:44:32,469 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Shard created, persistent : true 06:44:32,470 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards-notifier#-424254581 created and ready for shard:member-1-shard-pets-config_testAddReplicasForAllShards 06:44:32,470 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-pets-config_testAddReplicasForAllShards: Starting recovery with journal batch size 1 06:44:32,472 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-pets-config_testAddReplicasForAllShards: journal open: applyTo=0 06:44:32,473 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:32,473 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-pets-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:44:32,476 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-pets-config_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 06:44:32,477 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 06:44:32,477 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:32,477 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , received role change from null to Follower 06:44:32,478 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [pets] 06:44:32,478 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-pets-config_testAddReplicasForAllShards is initialized 06:44:32,480 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-pets-config_testAddReplicasForAllShards (Candidate): Starting new election term 1 06:44:32,480 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-pets-config_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:44:32,480 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 06:44:32,480 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:32,480 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , received role change from Follower to Candidate 06:44:32,480 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-pets-config_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 06:44:32,480 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 06:44:32,480 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: became leader with frontend state for [] 06:44:32,480 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , received role change from Candidate to Leader 06:44:32,481 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 06:44:32,481 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@74fc0391 06:44:32,481 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testAddReplicasForAllShards from Candidate to Leader 06:44:32,481 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddReplicasForAllShards is ready 06:44:32,525 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#598703368] 06:44:32,525 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : pets 06:44:32,525 PM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#-662370582] 06:44:32,542 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 06:44:32,570 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-1169309409323467805] 06:44:32,570 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 06:44:32,576 PM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:44:32,576 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 06:44:32,577 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:44:32,577 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:44:32,607 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:32,617 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:32,622 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddReplicasForAllShards 06:44:32,632 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-1491524896] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:32,633 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 06:44:32,633 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 06:44:32,641 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 06:44:32,642 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 06:44:32,644 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@29c19302} 06:44:32,644 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddReplicasForAllShards, generation=0} started 06:44:32,645 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:44:32,645 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:32,648 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:32,650 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddReplicasForAllShards 06:44:32,652 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 06:44:32,652 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 06:44:32,656 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#1923259225] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:32,665 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 06:44:32,669 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 06:44:32,671 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} started 06:44:32,676 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@2706684a} 06:44:32,676 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:44:32,947 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 06:44:32,999 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 06:44:33,6 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:44:33,6 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 06:44:33,6 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:44:33,6 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddReplicasForAllShards is ready 06:44:33,196 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 06:44:33,197 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:44:33,197 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-pets-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards 06:44:33,197 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-pets-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards 06:44:33,197 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards 06:44:33,197 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 06:44:33,198 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards 06:44:33,198 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 06:44:33,198 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddReplicasForAllShards is ready 06:44:33,198 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:44:33,199 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards 06:44:33,199 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 06:44:33,199 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards 06:44:33,199 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 06:44:33,199 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddReplicasForAllShards is ready 06:44:33,286 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@42287a] 06:44:33,288 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2 doCreateShard: shardId: member-2-shard-pets-config_testAddReplicasForAllShards, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: false 06:44:33,290 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=no-leader-ns, moduleName=no-leader-module, shardName=no-leader, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@5ea83c4c] 06:44:33,290 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Shard created, persistent : true 06:44:33,290 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2 doCreateShard: shardId: member-2-shard-no-leader-oper_testAddReplicasForAllShards, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: false 06:44:33,292 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-2-shard-pets-config_testAddReplicasForAllShards: Starting recovery with journal batch size 1 06:44:33,292 PM [main] [INFO] ClusterAdminRpcService - Adding replicas for all shards 06:44:33,292 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: Shard created, persistent : true 06:44:33,292 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people, pets] for data store config_testAddReplicasForAllShards 06:44:33,293 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 06:44:33,294 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards-notifier#-1153872304 created and ready for shard:member-2-shard-no-leader-oper_testAddReplicasForAllShards 06:44:33,294 PM [cluster-test-shard-dispatcher-31] [INFO] RaftActor - member-2-shard-no-leader-oper_testAddReplicasForAllShards: Starting recovery with journal batch size 1 06:44:33,294 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards#1508814962] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:33,294 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards#-343384591] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:33,294 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards-notifier#-1153872304] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:33,294 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-notifier#1680569962 created and ready for shard:member-2-shard-pets-config_testAddReplicasForAllShards 06:44:33,295 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-notifier#1680569962] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:33,295 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-2-shard-pets-config_testAddReplicasForAllShards: journal open: applyTo=0 06:44:33,296 PM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-2-shard-no-leader-oper_testAddReplicasForAllShards: journal open: applyTo=0 06:44:33,297 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:33,298 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:33,298 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-2-shard-pets-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:44:33,298 PM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-2-shard-no-leader-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:44:33,297 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 06:44:33,300 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people, no-leader] for data store oper_testAddReplicasForAllShards 06:44:33,300 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 06:44:33,301 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=people] 06:44:33,301 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=no-leader] 06:44:33,302 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 06:44:33,302 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards] 06:44:33,302 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-2-shard-pets-config_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 06:44:33,303 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 06:44:33,303 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=people] 06:44:33,303 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:33,303 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testAddReplicasForAllShards , received role change from null to Follower 06:44:33,303 PM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-2-shard-no-leader-oper_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 06:44:33,303 PM [cluster-test-shard-dispatcher-11] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 06:44:33,304 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-no-leader-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 06:44:33,304 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:33,304 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-no-leader-oper_testAddReplicasForAllShards , received role change from null to Follower 06:44:33,304 PM [cluster-test-shard-dispatcher-31] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 06:44:33,304 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards] 06:44:33,305 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 06:44:33,305 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for people forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards] 06:44:33,305 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=no-leader, waitUntilReady=true] 06:44:33,305 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for no-leader forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards] 06:44:33,305 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [no-leader] 06:44:33,305 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-2-shard-no-leader-oper_testAddReplicasForAllShards is initialized 06:44:33,306 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=pets] 06:44:33,306 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [pets] 06:44:33,306 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-no-leader-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 06:44:33,306 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-pets-config_testAddReplicasForAllShards is initialized 06:44:33,306 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-no-leader-oper_testAddReplicasForAllShards from null to Follower 06:44:33,306 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 06:44:33,306 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 06:44:33,306 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for people forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards] 06:44:33,307 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=pets, waitUntilReady=true] 06:44:33,307 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for pets forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards] 06:44:33,313 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 06:44:33,313 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 06:44:33,314 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#962045525, primaryVersion=13] 06:44:33,314 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-1070578412, primaryVersion=13] 06:44:33,315 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=people, waitUntilReady=true] 06:44:33,315 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=people, waitUntilReady=true] 06:44:33,315 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for people: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#49494200, primaryVersion=13] 06:44:33,315 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for people: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#-1673069384, primaryVersion=13] 06:44:33,315 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=no-leader, waitUntilReady=true] 06:44:33,315 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: No shard found for no-leader 06:44:33,317 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=pets, waitUntilReady=true] 06:44:33,317 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for pets: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#-662370582, primaryVersion=13] 06:44:33,316 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-pets-config_testAddReplicasForAllShards from null to Follower 06:44:33,332 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-1070578412 for shard member-2-shard-cars-config_testAddReplicasForAllShards 06:44:33,332 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#962045525 for shard member-2-shard-cars-oper_testAddReplicasForAllShards 06:44:33,333 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Shard created, persistent : true 06:44:33,334 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-2-shard-cars-config_testAddReplicasForAllShards: Starting recovery with journal batch size 1 06:44:33,334 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Shard created, persistent : true 06:44:33,335 PM [cluster-test-shard-dispatcher-31] [INFO] RaftActor - member-2-shard-cars-oper_testAddReplicasForAllShards: Starting recovery with journal batch size 1 06:44:33,335 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testAddReplicasForAllShards: journal open: applyTo=0 06:44:33,336 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#49494200 for shard member-2-shard-people-oper_testAddReplicasForAllShards 06:44:33,336 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards#-1009325105] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:33,336 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards#285372669] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:33,336 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-notifier#-1960851397 created and ready for shard:member-2-shard-cars-config_testAddReplicasForAllShards 06:44:33,336 PM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testAddReplicasForAllShards: journal open: applyTo=0 06:44:33,336 PM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-2: Failed to find leader for shard no-leader: received response: {} org.opendaylight.controller.cluster.datastore.exceptions.PrimaryNotFoundException: No primary shard found for no-leader. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.findPrimary(ShardManager.java:1034) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:229) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 06:44:33,337 PM [cluster-test-pekko.actor.default-dispatcher-19] [WARN] ClusterAdminRpcService - Failed to add replica for shard ShardName{value=no-leader}, type Operational org.opendaylight.controller.cluster.datastore.exceptions.PrimaryNotFoundException: No primary shard found for no-leader. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.findPrimary(ShardManager.java:1034) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:229) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 06:44:33,338 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#-1673069384 for shard member-2-shard-people-config_testAddReplicasForAllShards 06:44:33,339 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#-662370582 for shard member-2-shard-pets-config_testAddReplicasForAllShards 06:44:33,336 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-notifier#-701530663 created and ready for shard:member-2-shard-cars-oper_testAddReplicasForAllShards 06:44:33,340 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Shard created, persistent : true 06:44:33,337 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Shard created, persistent : true 06:44:33,340 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards-notifier#-25197298 created and ready for shard:member-2-shard-people-oper_testAddReplicasForAllShards 06:44:33,336 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-notifier#-701530663] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:33,340 PM [cluster-test-shard-dispatcher-31] [INFO] RaftActor - member-2-shard-people-oper_testAddReplicasForAllShards: Starting recovery with journal batch size 1 06:44:33,341 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-notifier#-1960851397] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:33,343 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards-notifier#1652687709 created and ready for shard:member-2-shard-people-config_testAddReplicasForAllShards 06:44:33,343 PM [cluster-test-shard-dispatcher-32] [INFO] RaftActor - member-2-shard-people-config_testAddReplicasForAllShards: Starting recovery with journal batch size 1 06:44:33,344 PM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 1 06:44:33,345 PM [cluster-test-shard-dispatcher-32] [INFO] EnabledRaftStorage - member-2-shard-people-config_testAddReplicasForAllShards: journal open: applyTo=0 06:44:33,350 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:33,351 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:33,351 PM [cluster-test-shard-dispatcher-32] [INFO] PekkoRecovery - member-2-shard-cars-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:44:33,351 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:33,347 PM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-people-oper_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-people-oper_testAddReplicasForAllShards 06:44:33,347 PM [cluster-test-shard-dispatcher-19] [INFO] SnapshotManager - member-1-shard-cars-config_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-config_testAddReplicasForAllShards 06:44:33,351 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-2-shard-people-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:44:33,347 PM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-oper_testAddReplicasForAllShards 06:44:33,351 PM [cluster-test-shard-dispatcher-33] [INFO] PekkoRecovery - member-2-shard-cars-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:44:33,354 PM [cluster-test-shard-dispatcher-32] [INFO] PekkoRecovery - member-2-shard-cars-config_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 06:44:33,355 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 06:44:33,355 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testAddReplicasForAllShards is initialized 06:44:33,355 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-2-shard-people-config_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 06:44:33,355 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 06:44:33,355 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:33,356 PM [cluster-test-shard-dispatcher-33] [INFO] PekkoRecovery - member-2-shard-cars-oper_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 06:44:33,356 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 06:44:33,356 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testAddReplicasForAllShards is initialized 06:44:33,356 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 06:44:33,356 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:33,356 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 06:44:33,356 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testAddReplicasForAllShards is initialized 06:44:33,356 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 06:44:33,356 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testAddReplicasForAllShards , received role change from null to Follower 06:44:33,356 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddReplicasForAllShards , received role change from null to Follower 06:44:33,356 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 06:44:33,357 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 06:44:33,357 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:33,357 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testAddReplicasForAllShards from null to Follower 06:44:33,357 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddReplicasForAllShards , received role change from null to Follower 06:44:33,357 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testAddReplicasForAllShards from null to Follower 06:44:33,357 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 06:44:33,357 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testAddReplicasForAllShards from null to Follower 06:44:33,360 PM [cluster-test-shard-dispatcher-30] [INFO] SnapshotManager - member-1-shard-people-config_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-people-config_testAddReplicasForAllShards 06:44:33,360 PM [cluster-test-shard-dispatcher-31] [INFO] SnapshotManager - member-1-shard-pets-config_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-pets-config_testAddReplicasForAllShards 06:44:33,363 PM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testAddReplicasForAllShards: journal open: applyTo=0 06:44:33,366 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:33,366 PM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-2-shard-people-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:44:33,368 PM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-2-shard-people-oper_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 06:44:33,369 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 06:44:33,369 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:33,369 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testAddReplicasForAllShards , received role change from null to Follower 06:44:33,369 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 06:44:33,369 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testAddReplicasForAllShards is initialized 06:44:33,369 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 06:44:33,370 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testAddReplicasForAllShards from null to Follower 06:44:33,370 PM [cluster-test-shard-dispatcher-30] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddReplicasForAllShards: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 06:44:33,370 PM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-1-shard-cars-config_testAddReplicasForAllShards: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 06:44:33,371 PM [cluster-test-shard-dispatcher-31] [INFO] SnapshotManager - member-1-shard-pets-config_testAddReplicasForAllShards: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 06:44:33,373 PM [cluster-test-shard-dispatcher-30] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 06:44:33,373 PM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-1-shard-cars-config_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 06:44:33,373 PM [cluster-test-shard-dispatcher-31] [INFO] SnapshotManager - member-1-shard-pets-config_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 06:44:33,374 PM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-people-oper_testAddReplicasForAllShards: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 06:44:33,374 PM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-people-oper_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 06:44:33,376 PM [cluster-test-shard-dispatcher-19] [INFO] SnapshotManager - member-1-shard-people-config_testAddReplicasForAllShards: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 06:44:33,376 PM [cluster-test-shard-dispatcher-19] [INFO] SnapshotManager - member-1-shard-people-config_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 06:44:33,384 PM [cluster-test-shard-dispatcher-32] [INFO] Follower - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-people-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 06:44:33,385 PM [cluster-test-shard-dispatcher-31] [INFO] Follower - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 06:44:33,383 PM [cluster-test-shard-dispatcher-33] [INFO] Follower - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 06:44:33,385 PM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 06:44:33,385 PM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 06:44:33,391 PM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Snapshot received from leader: member-1-shard-cars-oper_testAddReplicasForAllShards 06:44:33,391 PM [cluster-test-shard-dispatcher-31] [INFO] Follower - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Snapshot received from leader: member-1-shard-cars-config_testAddReplicasForAllShards 06:44:33,391 PM [cluster-test-shard-dispatcher-32] [INFO] Follower - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Snapshot received from leader: member-1-shard-people-config_testAddReplicasForAllShards 06:44:33,391 PM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Snapshot received from leader: member-1-shard-pets-config_testAddReplicasForAllShards 06:44:33,392 PM [cluster-test-shard-dispatcher-33] [INFO] Follower - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Snapshot received from leader: member-1-shard-people-oper_testAddReplicasForAllShards 06:44:33,392 PM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-2-shard-pets-config_testAddReplicasForAllShards: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 06:44:33,392 PM [cluster-test-shard-dispatcher-32] [INFO] SnapshotManager - member-2-shard-people-config_testAddReplicasForAllShards: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 06:44:33,392 PM [cluster-test-shard-dispatcher-33] [INFO] SnapshotManager - member-2-shard-people-oper_testAddReplicasForAllShards: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 06:44:33,394 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddReplicasForAllShards status sync done false 06:44:33,394 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddReplicasForAllShards status sync done false 06:44:33,394 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testAddReplicasForAllShards status sync done false 06:44:33,394 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@42777185 06:44:33,394 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6cf3e3f9 06:44:33,395 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@528afd41 06:44:33,395 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testAddReplicasForAllShards status sync done false 06:44:33,395 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3bbe7186 06:44:33,395 PM [cluster-test-shard-dispatcher-31] [INFO] SnapshotManager - member-2-shard-cars-config_testAddReplicasForAllShards: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 06:44:33,395 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testAddReplicasForAllShards status sync done false 06:44:33,395 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3e86617e 06:44:33,395 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 06:44:33,397 PM [cluster-test-shard-dispatcher-30] [INFO] SnapshotManager - member-2-shard-cars-oper_testAddReplicasForAllShards: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 06:44:33,405 PM [cluster-test-shard-dispatcher-30] [INFO] SnapshotManager - member-1-shard-people-oper_testAddReplicasForAllShards: snapshot is durable as of 2025-08-07T18:44:33.374638279Z 06:44:33,406 PM [cluster-test-shard-dispatcher-31] [INFO] SnapshotManager - member-1-shard-cars-config_testAddReplicasForAllShards: snapshot is durable as of 2025-08-07T18:44:33.373303153Z 06:44:33,409 PM [cluster-test-shard-dispatcher-35] [INFO] SnapshotManager - member-2-shard-cars-config_testAddReplicasForAllShards: snapshot is durable as of 2025-08-07T18:44:33.395691297Z 06:44:33,410 PM [cluster-test-shard-dispatcher-31] [INFO] SnapshotManager - member-1-shard-people-config_testAddReplicasForAllShards: snapshot is durable as of 2025-08-07T18:44:33.376143490Z 06:44:33,412 PM [cluster-test-shard-dispatcher-31] [INFO] SnapshotManager - member-1-shard-pets-config_testAddReplicasForAllShards: snapshot is durable as of 2025-08-07T18:44:33.373064898Z 06:44:33,412 PM [cluster-test-shard-dispatcher-30] [INFO] SnapshotManager - member-2-shard-cars-oper_testAddReplicasForAllShards: snapshot is durable as of 2025-08-07T18:44:33.397578255Z 06:44:33,412 PM [cluster-test-shard-dispatcher-30] [INFO] ShardSnapshotCohort - member-2-shard-cars-oper_testAddReplicasForAllShards: Applying snapshot 06:44:33,412 PM [cluster-test-shard-dispatcher-35] [INFO] ShardSnapshotCohort - member-2-shard-cars-config_testAddReplicasForAllShards: Applying snapshot 06:44:33,413 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 06:44:33,414 PM [cluster-test-shard-dispatcher-34] [INFO] SnapshotManager - member-2-shard-people-oper_testAddReplicasForAllShards: snapshot is durable as of 2025-08-07T18:44:33.394569814Z 06:44:33,414 PM [cluster-test-shard-dispatcher-34] [INFO] ShardSnapshotCohort - member-2-shard-people-oper_testAddReplicasForAllShards: Applying snapshot 06:44:33,414 PM [cluster-test-shard-dispatcher-34] [DEBUG] FrontendMetadata - member-2-shard-people-oper_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 06:44:33,413 PM [cluster-test-shard-dispatcher-35] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 06:44:33,415 PM [cluster-test-shard-dispatcher-33] [INFO] SnapshotManager - member-2-shard-pets-config_testAddReplicasForAllShards: snapshot is durable as of 2025-08-07T18:44:33.394570844Z 06:44:33,415 PM [cluster-test-shard-dispatcher-33] [INFO] ShardSnapshotCohort - member-2-shard-pets-config_testAddReplicasForAllShards: Applying snapshot 06:44:33,415 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendMetadata - member-2-shard-pets-config_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 06:44:33,415 PM [cluster-test-shard-dispatcher-31] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddReplicasForAllShards: snapshot is durable as of 2025-08-07T18:44:33.373064218Z 06:44:33,417 PM [cluster-test-shard-dispatcher-32] [INFO] SnapshotManager - member-2-shard-people-config_testAddReplicasForAllShards: snapshot is durable as of 2025-08-07T18:44:33.394572774Z 06:44:33,417 PM [cluster-test-shard-dispatcher-32] [INFO] ShardSnapshotCohort - member-2-shard-people-config_testAddReplicasForAllShards: Applying snapshot 06:44:33,417 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-2-shard-people-config_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 06:44:33,439 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-oper_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher#-1387050595] with name member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher 06:44:33,439 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-config_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher#1156972260] with name member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher 06:44:33,439 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-pets-config_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher#-2128661340] with name member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher 06:44:33,440 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-oper_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher#1022114149] with name member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher 06:44:33,440 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-people-config_testAddReplicasForAllShards: state snapshot applied in 23.54 ms 06:44:33,440 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-2-shard-people-oper_testAddReplicasForAllShards: state snapshot applied in 26.70 ms 06:44:33,441 PM [cluster-test-shard-dispatcher-32] [INFO] ShardSnapshotCohort - member-2-shard-people-config_testAddReplicasForAllShards: Done applying snapshot 06:44:33,440 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-2-shard-pets-config_testAddReplicasForAllShards: state snapshot applied in 25.43 ms 06:44:33,441 PM [cluster-test-shard-dispatcher-34] [INFO] ShardSnapshotCohort - member-2-shard-people-oper_testAddReplicasForAllShards: Done applying snapshot 06:44:33,441 PM [cluster-test-shard-dispatcher-33] [INFO] ShardSnapshotCohort - member-2-shard-pets-config_testAddReplicasForAllShards: Done applying snapshot 06:44:33,442 PM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher#-2128661340] 06:44:33,443 PM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher#1022114149] 06:44:33,444 PM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testAddReplicasForAllShards: publishChanges: rootPath: / 06:44:33,444 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher#1156972260] 06:44:33,445 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-config_testAddReplicasForAllShards: publishChanges: rootPath: / 06:44:33,446 PM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher#-1387050595] 06:44:33,446 PM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-oper_testAddReplicasForAllShards: publishChanges: rootPath: / 06:44:33,447 PM [cluster-test-shard-dispatcher-30] [INFO] AbstractLeader - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-people-config_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 06:44:33,448 PM [cluster-test-shard-dispatcher-31] [INFO] AbstractLeader - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-people-oper_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 06:44:33,441 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testAddReplicasForAllShards: state snapshot applied in 35.45 ms 06:44:33,448 PM [cluster-test-shard-dispatcher-30] [INFO] ShardSnapshotCohort - member-2-shard-cars-oper_testAddReplicasForAllShards: Done applying snapshot 06:44:33,449 PM [cluster-test-shard-dispatcher-12] [INFO] AbstractLeader - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-pets-config_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 06:44:33,450 PM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher#2129394859] 06:44:33,450 PM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-pets-config_testAddReplicasForAllShards: publishChanges: rootPath: / 06:44:33,450 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-config_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher#2129394859] with name member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher 06:44:33,450 PM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testAddReplicasForAllShards: publishChanges: rootPath: / 06:44:33,450 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardDataTree - member-2-shard-cars-config_testAddReplicasForAllShards: state snapshot applied in 37.41 ms 06:44:33,450 PM [cluster-test-shard-dispatcher-35] [INFO] ShardSnapshotCohort - member-2-shard-cars-config_testAddReplicasForAllShards: Done applying snapshot 06:44:33,453 PM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher: 3.006 ms 06:44:33,453 PM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-pets-config_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher: 3.264 ms 06:44:33,453 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-config_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher: 8.585 ms 06:44:33,453 PM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-oper_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher: 6.917 ms 06:44:33,459 PM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher: 15.05 ms 06:44:33,461 PM [cluster-test-shard-dispatcher-19] [INFO] AbstractLeader - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-cars-oper_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 06:44:33,462 PM [cluster-test-shard-dispatcher-11] [INFO] AbstractLeader - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-cars-config_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 06:44:33,475 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1873854a 06:44:33,475 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@39bf3505 06:44:33,475 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5d22b0d1 06:44:33,476 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 06:44:33,476 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testAddReplicasForAllShards} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#962045525 06:44:33,476 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard cars 06:44:33,476 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-oper_testAddReplicasForAllShards 06:44:33,476 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars] 06:44:33,478 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@15135bff 06:44:33,478 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 06:44:33,478 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testAddReplicasForAllShards} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-1070578412 06:44:33,478 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard cars 06:44:33,479 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-config_testAddReplicasForAllShards 06:44:33,479 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [cars] 06:44:33,483 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 06:44:33,483 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-people-oper_testAddReplicasForAllShards} for shard people from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#49494200 06:44:33,483 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard people 06:44:33,483 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 06:44:33,483 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:33,483 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-people-oper_testAddReplicasForAllShards 06:44:33,483 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:33,483 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars, people] 06:44:33,483 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 06:44:33,484 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:33,484 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-pets-config_testAddReplicasForAllShards} for shard pets from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#-662370582 06:44:33,484 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard pets 06:44:33,484 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-pets-config_testAddReplicasForAllShards 06:44:33,484 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [pets, cars] 06:44:33,484 PM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 06:44:33,484 PM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 2 06:44:33,484 PM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 06:44:33,484 PM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 3 06:44:33,482 PM [cluster-test-shard-dispatcher-30] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-oper_testAddReplicasForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 06:44:33,486 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 06:44:33,486 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:33,486 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testAddReplicasForAllShards status sync done true 06:44:33,486 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddReplicasForAllShards status sync done true 06:44:33,486 PM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 06:44:33,486 PM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 4 06:44:33,487 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testAddReplicasForAllShards status sync done false 06:44:33,488 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-people-config_testAddReplicasForAllShards} for shard people from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#-1673069384 06:44:33,488 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard people 06:44:33,488 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-people-config_testAddReplicasForAllShards 06:44:33,488 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [pets, cars, people] 06:44:33,489 PM [cluster-test-shard-dispatcher-30] [INFO] RaftActorVotingConfigSupport - member-1-shard-pets-config_testAddReplicasForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 06:44:33,489 PM [cluster-test-shard-dispatcher-31] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 06:44:33,489 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 06:44:33,489 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:33,489 PM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=pets}, type Config 06:44:33,489 PM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 5 06:44:33,489 PM [cluster-test-shard-dispatcher-31] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testAddReplicasForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 06:44:33,490 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@61f67bb6 06:44:33,490 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 06:44:33,490 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testAddReplicasForAllShards status sync done true 06:44:33,490 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddReplicasForAllShards status sync done true 06:44:33,490 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testAddReplicasForAllShards status sync done true 06:44:33,490 PM [cluster-test-shard-dispatcher-31] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-config_testAddReplicasForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 06:44:33,491 PM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 06:44:33,491 PM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 6 06:44:33,498 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testAddReplicasForAllShards status sync done false 06:44:33,498 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddReplicasForAllShards status sync done false 06:44:33,498 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddReplicasForAllShards status sync done false 06:44:33,499 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testAddReplicasForAllShards status sync done false 06:44:33,501 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-1491524896] 06:44:33,502 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 06:44:33,504 PM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards#-1009325105] 06:44:33,507 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards#-1009325105] 06:44:33,507 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-1491524896] 06:44:33,508 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 06:44:33,508 PM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards#-1299334725] 06:44:33,508 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards#-1299334725] 06:44:33,508 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-1491524896] 06:44:33,508 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : pets 06:44:33,508 PM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards#1508814962] 06:44:33,508 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards#1508814962] 06:44:33,508 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#1923259225] 06:44:33,509 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 06:44:33,509 PM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards#285372669] 06:44:33,509 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards#285372669] 06:44:33,509 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#1923259225] 06:44:33,509 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 06:44:33,509 PM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards#-1891896204] 06:44:33,509 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards#-1891896204] 06:44:33,510 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 06:44:33,510 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-pets-config_testAddReplicasForAllShards 06:44:33,510 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testAddReplicasForAllShards 06:44:33,510 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testAddReplicasForAllShards 06:44:33,510 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 3 shards 06:44:33,511 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Stopping Shard 06:44:33,511 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Stopping Shard 06:44:33,511 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Stopping Shard 06:44:33,512 PM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testAddReplicasForAllShards: journal closed 06:44:33,512 PM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-2-shard-pets-config_testAddReplicasForAllShards: journal closed 06:44:33,512 PM [cluster-test-shard-dispatcher-34] [INFO] EnabledRaftStorage - member-2-shard-people-config_testAddReplicasForAllShards: journal closed 06:44:33,512 PM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 06:44:33,513 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 06:44:33,513 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 06:44:33,513 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testAddReplicasForAllShards 06:44:33,514 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testAddReplicasForAllShards 06:44:33,514 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-no-leader-oper_testAddReplicasForAllShards 06:44:33,514 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 3 shards 06:44:33,514 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Stopping Shard 06:44:33,514 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Stopping Shard 06:44:33,514 PM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testAddReplicasForAllShards: journal closed 06:44:33,514 PM [cluster-test-shard-dispatcher-34] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testAddReplicasForAllShards: journal closed 06:44:33,514 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: Stopping Shard 06:44:33,515 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-2-shard-no-leader-oper_testAddReplicasForAllShards: journal closed 06:44:33,515 PM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 06:44:33,515 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 06:44:33,517 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:44:33,517 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:44:33,540 PM [cluster-test-pekko.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:44:33,541 PM [cluster-test-pekko.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:44:33,551 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 06:44:33,571 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 06:44:33,573 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-pets-config_testAddReplicasForAllShards 06:44:33,573 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testAddReplicasForAllShards 06:44:33,573 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testAddReplicasForAllShards 06:44:33,574 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 3 shards 06:44:33,580 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7c971a45 06:44:33,580 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7d9f9e0b 06:44:33,581 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@58047f8f 06:44:33,581 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2087740c 06:44:33,581 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddReplicasForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2087740c 06:44:33,581 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddReplicasForAllShards: pauseLeader successfully completed - doing transfer 06:44:33,581 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-config_testAddReplicasForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@58047f8f 06:44:33,582 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testAddReplicasForAllShards: pauseLeader successfully completed - doing transfer 06:44:33,583 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@277bcf7a 06:44:33,583 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-pets-config_testAddReplicasForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@277bcf7a 06:44:33,583 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testAddReplicasForAllShards: pauseLeader successfully completed - doing transfer 06:44:33,583 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6749567b 06:44:34,548 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-1169309409323467805] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 06:44:37,831 PM [cluster-test-pekko.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 06:44:38,151 PM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 06:44:38,311 PM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 06:44:38,470 PM [cluster-test-pekko.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 06:44:39,294 PM [cluster-test-pekko.actor.default-dispatcher-18] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2559, Up)]. 06:44:39,297 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:44:39,297 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:44:40,72 PM [cluster-test-pekko.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 06:44:40,348 PM [cluster-test-pekko.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 06:44:41,29 PM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 06:44:41,349 PM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 06:44:41,367 PM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 06:44:41,990 PM [cluster-test-pekko.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 06:44:42,296 PM [cluster-test-pekko.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 06:44:42,389 PM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 06:44:42,618 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddReplicasForAllShards: leader transfer aborted 06:44:42,618 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testAddReplicasForAllShards: leader transfer aborted 06:44:42,618 PM [cluster-test-shard-dispatcher-12] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddReplicasForAllShards: Failed to transfer leadership in 9.038 s 06:44:42,618 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: In unpauseLeader 06:44:42,618 PM [cluster-test-shard-dispatcher-30] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testAddReplicasForAllShards: Failed to transfer leadership in 9.035 s 06:44:42,618 PM [cluster-test-shard-dispatcher-31] [WARN] Leader - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): At least 1 followers need to be active, Switching member-1-shard-people-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 06:44:42,618 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: In unpauseLeader 06:44:42,619 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testAddReplicasForAllShards: leader transfer aborted 06:44:42,619 PM [cluster-test-shard-dispatcher-30] [WARN] Leader - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): At least 1 followers need to be active, Switching member-1-shard-pets-config_testAddReplicasForAllShards from Leader to IsolatedLeader 06:44:42,619 PM [cluster-test-shard-dispatcher-12] [WARN] Leader - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-config_testAddReplicasForAllShards from Leader to IsolatedLeader 06:44:42,619 PM [cluster-test-shard-dispatcher-31] [INFO] RaftActorBehavior - member-1-shard-people-oper_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 06:44:42,619 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 06:44:42,619 PM [cluster-test-shard-dispatcher-19] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testAddReplicasForAllShards: Failed to transfer leadership in 9.039 s 06:44:42,619 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: In unpauseLeader 06:44:42,619 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 06:44:42,619 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 06:44:42,619 PM [cluster-test-shard-dispatcher-30] [INFO] RaftActorBehavior - member-1-shard-pets-config_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 06:44:42,619 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 06:44:42,619 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 06:44:42,619 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 06:44:42,620 PM [cluster-test-shard-dispatcher-19] [WARN] Leader - member-1-shard-people-config_testAddReplicasForAllShards (Leader): At least 1 followers need to be active, Switching member-1-shard-people-config_testAddReplicasForAllShards from Leader to IsolatedLeader 06:44:42,620 PM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-people-config_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 06:44:42,620 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 06:44:42,621 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Stopping Shard 06:44:42,621 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Stopping Shard 06:44:42,621 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Stopping Shard 06:44:42,621 PM [cluster-test-shard-dispatcher-11] [WARN] Leader - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 06:44:42,622 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 06:44:42,622 PM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testAddReplicasForAllShards: journal closed 06:44:42,622 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-people-config_testAddReplicasForAllShards: journal closed 06:44:42,622 PM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-1-shard-pets-config_testAddReplicasForAllShards: journal closed 06:44:42,622 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 06:44:42,622 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 06:44:42,622 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 06:44:42,623 PM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 06:44:42,623 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 06:44:42,624 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 06:44:42,625 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testAddReplicasForAllShards 06:44:42,625 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testAddReplicasForAllShards 06:44:42,625 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 06:44:42,626 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Stopping Shard 06:44:42,626 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Stopping Shard 06:44:42,626 PM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testAddReplicasForAllShards: journal closed 06:44:42,627 PM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testAddReplicasForAllShards: journal closed 06:44:42,627 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 06:44:42,628 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 06:44:42,631 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:44:42,631 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:44:42,637 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/07/2025 18:44:42.640] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:44:42,684 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 06:44:42,703 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [5047794493267531968] 06:44:42,704 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 06:44:42,708 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 06:44:42,709 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 06:44:42,711 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:44:42,711 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:44:42,733 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 06:44:42,733 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 06:44:42,734 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 06:44:42,734 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:42,736 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:42,739 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStates 06:44:42,748 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 06:44:42,749 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 06:44:42,749 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#427315978] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:42,755 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 06:44:42,756 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testFlipMemberVotingStates 06:44:42,757 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testFlipMemberVotingStates 06:44:42,757 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:44:42,757 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 06:44:42,757 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 06:44:42,761 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 06:44:42,762 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@660f730a} 06:44:42,762 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} started 06:44:42,763 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:42,765 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:42,766 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:44:42,766 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testFlipMemberVotingStates 06:44:42,768 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Shard created, persistent : true 06:44:42,769 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-config_testFlipMemberVotingStates: Starting recovery with journal batch size 1 06:44:42,771 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testFlipMemberVotingStates: journal open: applyTo=0 06:44:42,771 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 06:44:42,771 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 06:44:42,771 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testFlipMemberVotingStates on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates on shard member-1-shard-cars-config_testFlipMemberVotingStates whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-737304395] 06:44:42,771 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testFlipMemberVotingStates 06:44:42,771 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 06:44:42,771 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 06:44:42,771 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testFlipMemberVotingStates on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates on shard member-1-shard-people-config_testFlipMemberVotingStates whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#-1957251928] 06:44:42,772 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Shard created, persistent : true 06:44:42,773 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-1-shard-people-config_testFlipMemberVotingStates: Starting recovery with journal batch size 1 06:44:42,775 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStates 06:44:42,776 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-people-config_testFlipMemberVotingStates: journal open: applyTo=0 06:44:42,776 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates-notifier#-1472966702 created and ready for shard:member-1-shard-people-config_testFlipMemberVotingStates 06:44:42,776 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-737304395] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:42,777 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates-notifier#824193374 created and ready for shard:member-1-shard-cars-config_testFlipMemberVotingStates 06:44:42,778 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 06:44:42,778 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 06:44:42,779 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 06:44:42,779 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testFlipMemberVotingStates 06:44:42,780 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testFlipMemberVotingStates 06:44:42,780 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:44:42,780 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 06:44:42,780 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 06:44:42,781 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#-1957251928] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:42,782 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates-notifier#-1472966702] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:42,782 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates-notifier#824193374] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:42,782 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1162703038] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:42,785 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 06:44:42,786 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} started 06:44:42,798 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:44:42,798 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:44:42,799 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:42,799 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:44:42,800 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:44:42,801 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:44:42,801 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:42,801 PM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-people-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:44:42,801 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 06:44:42,801 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 06:44:42,801 PM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-cars-config_testFlipMemberVotingStates: Taking snapshot on entry with index -1 06:44:42,803 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@4a49a87e} 06:44:42,803 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:44:42,803 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testFlipMemberVotingStates 06:44:42,804 PM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-people-config_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 06:44:42,804 PM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-people-config_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 06:44:42,804 PM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-1-shard-people-config_testFlipMemberVotingStates: Taking snapshot on entry with index -1 06:44:42,807 PM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-cars-config_testFlipMemberVotingStates: Snapshot completed in 6.137 ms, resetting timer for the next recovery snapshot 06:44:42,808 PM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-1-shard-people-config_testFlipMemberVotingStates: Snapshot completed in 3.792 ms, resetting timer for the next recovery snapshot 06:44:42,812 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:42,812 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:42,812 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 06:44:42,812 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testFlipMemberVotingStates is initialized 06:44:42,812 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 06:44:42,812 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testFlipMemberVotingStates is initialized 06:44:42,812 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from null to Follower 06:44:42,817 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 06:44:42,817 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 06:44:42,817 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testFlipMemberVotingStates on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates on shard member-1-shard-cars-oper_testFlipMemberVotingStates whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#602092281] 06:44:42,817 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testFlipMemberVotingStates 06:44:42,817 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 06:44:42,818 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 06:44:42,818 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 06:44:42,818 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testFlipMemberVotingStates on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates on shard member-1-shard-people-oper_testFlipMemberVotingStates whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#1427234982] 06:44:42,818 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Shard created, persistent : true 06:44:42,819 PM [cluster-test-shard-dispatcher-19] [INFO] RaftActor - member-1-shard-cars-oper_testFlipMemberVotingStates: Starting recovery with journal batch size 1 06:44:42,821 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates-notifier#-730740477 created and ready for shard:member-1-shard-cars-oper_testFlipMemberVotingStates 06:44:42,821 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#602092281] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:42,821 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates-notifier#-730740477] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:42,821 PM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testFlipMemberVotingStates: journal open: applyTo=0 06:44:42,822 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Shard created, persistent : true 06:44:42,823 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#1427234982] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:42,823 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates-notifier#-1519384174 created and ready for shard:member-1-shard-people-oper_testFlipMemberVotingStates 06:44:42,823 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates-notifier#-1519384174] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:42,824 PM [cluster-test-shard-dispatcher-19] [INFO] RaftActor - member-1-shard-people-oper_testFlipMemberVotingStates: Starting recovery with journal batch size 1 06:44:42,825 PM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testFlipMemberVotingStates: journal open: applyTo=0 06:44:42,825 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:42,825 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:42,825 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from null to Follower 06:44:42,826 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 06:44:42,826 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:44:42,827 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:44:42,828 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:44:42,828 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:42,828 PM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-1-shard-people-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:44:42,832 PM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-1-shard-people-oper_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 06:44:42,832 PM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-1-shard-people-oper_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 06:44:42,832 PM [cluster-test-shard-dispatcher-19] [INFO] Recovery - member-1-shard-people-oper_testFlipMemberVotingStates: Taking snapshot on entry with index -1 06:44:42,832 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from null to Follower 06:44:42,832 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from null to Follower 06:44:42,832 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:44:42,833 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:42,833 PM [cluster-test-shard-dispatcher-24] [INFO] PekkoRecovery - member-1-shard-cars-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:44:42,834 PM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 06:44:42,835 PM [cluster-test-shard-dispatcher-24] [INFO] PekkoRecovery - member-1-shard-cars-oper_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 06:44:42,835 PM [cluster-test-shard-dispatcher-24] [INFO] PekkoRecovery - member-1-shard-cars-oper_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 06:44:42,835 PM [cluster-test-shard-dispatcher-24] [INFO] Recovery - member-1-shard-cars-oper_testFlipMemberVotingStates: Taking snapshot on entry with index -1 06:44:42,836 PM [cluster-test-shard-dispatcher-19] [INFO] Recovery - member-1-shard-people-oper_testFlipMemberVotingStates: Snapshot completed in 4.083 ms, resetting timer for the next recovery snapshot 06:44:42,842 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:42,842 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:42,842 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 06:44:42,842 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testFlipMemberVotingStates is initialized 06:44:42,843 PM [cluster-test-shard-dispatcher-24] [INFO] Recovery - member-1-shard-cars-oper_testFlipMemberVotingStates: Snapshot completed in 7.572 ms, resetting timer for the next recovery snapshot 06:44:42,844 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from null to Follower 06:44:42,844 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 06:44:42,844 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from null to Follower 06:44:42,850 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 06:44:42,850 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testFlipMemberVotingStates is initialized 06:44:42,850 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:42,850 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:42,850 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from null to Follower 06:44:42,851 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 06:44:42,851 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from null to Follower 06:44:42,859 PM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-8425085304665297547] 06:44:42,860 PM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 06:44:42,863 PM [cluster-test-pekko.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:44:42,863 PM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 06:44:42,864 PM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:44:42,865 PM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:44:42,879 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:42,888 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:42,891 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStates 06:44:42,905 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 06:44:42,905 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 06:44:42,912 PM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#75895194] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:42,915 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 06:44:42,915 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testFlipMemberVotingStates 06:44:42,916 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testFlipMemberVotingStates 06:44:42,918 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 06:44:42,919 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@53229471} 06:44:42,919 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStates, generation=0} started 06:44:42,920 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:42,921 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:44:42,921 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testFlipMemberVotingStates 06:44:42,923 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:42,926 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStates 06:44:42,934 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testFlipMemberVotingStates 06:44:42,936 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Shard created, persistent : true 06:44:42,936 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#-932454036] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:42,936 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates-notifier#-1780492992 created and ready for shard:member-2-shard-cars-config_testFlipMemberVotingStates 06:44:42,936 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-cars-config_testFlipMemberVotingStates: Starting recovery with journal batch size 1 06:44:42,936 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates-notifier#-1780492992] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:42,937 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testFlipMemberVotingStates: journal open: applyTo=0 06:44:42,938 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Shard created, persistent : true 06:44:42,938 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#-1889593918] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:42,939 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates-notifier#1015719721 created and ready for shard:member-2-shard-people-config_testFlipMemberVotingStates 06:44:42,939 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates-notifier#1015719721] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:42,942 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-1706716598] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:42,945 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 06:44:42,945 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 06:44:42,945 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:44:42,946 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-people-config_testFlipMemberVotingStates: Starting recovery with journal batch size 1 06:44:42,946 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:44:42,946 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:42,946 PM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-2-shard-cars-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:44:42,947 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-people-config_testFlipMemberVotingStates: journal open: applyTo=0 06:44:42,947 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 06:44:42,948 PM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-2-shard-cars-config_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 06:44:42,948 PM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-2-shard-cars-config_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 06:44:42,948 PM [cluster-test-shard-dispatcher-29] [INFO] Recovery - member-2-shard-cars-config_testFlipMemberVotingStates: Taking snapshot on entry with index -1 06:44:42,949 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} started 06:44:42,953 PM [cluster-test-shard-dispatcher-29] [INFO] Recovery - member-2-shard-cars-config_testFlipMemberVotingStates: Snapshot completed in 4.894 ms, resetting timer for the next recovery snapshot 06:44:42,954 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 06:44:42,954 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testFlipMemberVotingStates 06:44:42,955 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testFlipMemberVotingStates 06:44:42,955 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@6cc3bcb9} 06:44:42,955 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:44:42,955 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testFlipMemberVotingStates 06:44:42,956 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testFlipMemberVotingStates 06:44:42,956 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:44:42,957 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:44:42,957 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#-311609404] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:42,957 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:42,957 PM [cluster-test-shard-dispatcher-32] [INFO] PekkoRecovery - member-2-shard-people-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:44:42,957 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Shard created, persistent : true 06:44:42,957 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Shard created, persistent : true 06:44:42,958 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#-1374956192] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:42,958 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates-notifier#-1643150215 created and ready for shard:member-2-shard-cars-oper_testFlipMemberVotingStates 06:44:42,958 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates-notifier#-1643150215] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:42,958 PM [cluster-test-shard-dispatcher-33] [INFO] RaftActor - member-2-shard-cars-oper_testFlipMemberVotingStates: Starting recovery with journal batch size 1 06:44:42,959 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates-notifier#-342888067] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:42,959 PM [cluster-test-shard-dispatcher-28] [INFO] RaftActor - member-2-shard-people-oper_testFlipMemberVotingStates: Starting recovery with journal batch size 1 06:44:42,959 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates-notifier#-342888067 created and ready for shard:member-2-shard-people-oper_testFlipMemberVotingStates 06:44:42,960 PM [cluster-test-shard-dispatcher-32] [INFO] PekkoRecovery - member-2-shard-people-config_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 06:44:42,960 PM [cluster-test-shard-dispatcher-32] [INFO] PekkoRecovery - member-2-shard-people-config_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 06:44:42,960 PM [cluster-test-shard-dispatcher-32] [INFO] Recovery - member-2-shard-people-config_testFlipMemberVotingStates: Taking snapshot on entry with index -1 06:44:42,960 PM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testFlipMemberVotingStates: journal open: applyTo=0 06:44:42,960 PM [cluster-test-shard-dispatcher-33] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testFlipMemberVotingStates: journal open: applyTo=0 06:44:42,961 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 06:44:42,961 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testFlipMemberVotingStates is initialized 06:44:42,961 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:44:42,962 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:44:42,962 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:42,962 PM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-2-shard-people-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:44:42,964 PM [cluster-test-shard-dispatcher-32] [INFO] Recovery - member-2-shard-people-config_testFlipMemberVotingStates: Snapshot completed in 4.272 ms, resetting timer for the next recovery snapshot 06:44:42,964 PM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-2-shard-people-oper_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 06:44:42,964 PM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-2-shard-people-oper_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 06:44:42,964 PM [cluster-test-shard-dispatcher-28] [INFO] Recovery - member-2-shard-people-oper_testFlipMemberVotingStates: Taking snapshot on entry with index -1 06:44:42,968 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:44:42,968 PM [cluster-test-shard-dispatcher-28] [INFO] Recovery - member-2-shard-people-oper_testFlipMemberVotingStates: Snapshot completed in 4.188 ms, resetting timer for the next recovery snapshot 06:44:42,971 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 06:44:42,971 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testFlipMemberVotingStates is initialized 06:44:42,971 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:42,971 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:42,971 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , received role change from null to Follower 06:44:42,972 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 06:44:42,972 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStates from null to Follower 06:44:42,972 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:42,972 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:42,972 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , received role change from null to Follower 06:44:42,973 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 06:44:42,973 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testFlipMemberVotingStates is initialized 06:44:42,973 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 06:44:42,973 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStates from null to Follower 06:44:42,976 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:42,976 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:42,976 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , received role change from null to Follower 06:44:42,976 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 06:44:42,976 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStates from null to Follower 06:44:42,976 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:44:42,976 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:42,976 PM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-2-shard-cars-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:44:42,979 PM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-2-shard-cars-oper_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 06:44:42,979 PM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-2-shard-cars-oper_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 06:44:42,979 PM [cluster-test-shard-dispatcher-29] [INFO] Recovery - member-2-shard-cars-oper_testFlipMemberVotingStates: Taking snapshot on entry with index -1 06:44:42,982 PM [cluster-test-shard-dispatcher-29] [INFO] Recovery - member-2-shard-cars-oper_testFlipMemberVotingStates: Snapshot completed in 2.174 ms, resetting timer for the next recovery snapshot 06:44:42,987 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:42,987 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:42,987 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 06:44:42,987 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testFlipMemberVotingStates is initialized 06:44:42,987 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , received role change from null to Follower 06:44:42,987 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 06:44:42,987 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStates from null to Follower 06:44:42,992 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 06:44:42,998 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 06:44:43,0 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:44:43,0 PM [cluster-test-shard-dispatcher-36] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 06:44:43,0 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 06:44:43,0 PM [cluster-test-shard-dispatcher-36] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 06:44:43,0 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 06:44:43,0 PM [cluster-test-shard-dispatcher-36] [INFO] PeerInfos - member-2-shard-cars-config_testFlipMemberVotingStates: Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 06:44:43,0 PM [cluster-test-shard-dispatcher-36] [INFO] PeerInfos - member-2-shard-people-config_testFlipMemberVotingStates: Peer address for peer member-1-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 06:44:43,0 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:44:43,0 PM [cluster-test-shard-dispatcher-36] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 06:44:43,0 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 06:44:43,0 PM [cluster-test-shard-dispatcher-36] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 06:44:43,0 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 06:44:43,0 PM [cluster-test-shard-dispatcher-36] [INFO] PeerInfos - member-2-shard-cars-oper_testFlipMemberVotingStates: Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 06:44:43,0 PM [cluster-test-shard-dispatcher-36] [INFO] PeerInfos - member-2-shard-people-oper_testFlipMemberVotingStates: Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 06:44:43,2 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 06:44:43,16 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [4363116322889847620] 06:44:43,16 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 06:44:43,20 PM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:44:43,21 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 06:44:43,23 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:44:43,23 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:44:43,41 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:43,43 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:43,47 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStates 06:44:43,75 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#304900971] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:43,78 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 06:44:43,78 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 06:44:43,83 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 06:44:43,83 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 06:44:43,83 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testFlipMemberVotingStates 06:44:43,84 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testFlipMemberVotingStates 06:44:43,86 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 06:44:43,86 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 06:44:43,87 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} started 06:44:43,87 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:43,90 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:43,92 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStates 06:44:43,93 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@7c320308} 06:44:43,93 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:44:43,93 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testFlipMemberVotingStates 06:44:43,103 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 06:44:43,104 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} started 06:44:43,104 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 06:44:43,108 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Shard created, persistent : true 06:44:43,108 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#-1807816411] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:43,109 PM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates-notifier#3889716 created and ready for shard:member-3-shard-cars-config_testFlipMemberVotingStates 06:44:43,109 PM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates-notifier#3889716] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:43,109 PM [cluster-test-shard-dispatcher-27] [INFO] RaftActor - member-3-shard-cars-config_testFlipMemberVotingStates: Starting recovery with journal batch size 1 06:44:43,110 PM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testFlipMemberVotingStates: journal open: applyTo=0 06:44:43,110 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testFlipMemberVotingStates 06:44:43,112 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:44:43,112 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 06:44:43,112 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 06:44:43,112 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 06:44:43,112 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 06:44:43,113 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Shard created, persistent : true 06:44:43,113 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#2068627754] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:43,113 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:44:43,114 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#897167298] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:43,115 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 06:44:43,115 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 06:44:43,117 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:44:43,117 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:43,117 PM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-3-shard-cars-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:44:43,118 PM [cluster-test-shard-dispatcher-27] [INFO] RaftActor - member-3-shard-people-config_testFlipMemberVotingStates: Starting recovery with journal batch size 1 06:44:43,120 PM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-3-shard-cars-config_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 06:44:43,120 PM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-3-shard-cars-config_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 06:44:43,120 PM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-3-shard-cars-config_testFlipMemberVotingStates: Taking snapshot on entry with index -1 06:44:43,121 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates-notifier#2132439526 created and ready for shard:member-3-shard-people-config_testFlipMemberVotingStates 06:44:43,121 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates-notifier#2132439526] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:43,121 PM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-3-shard-people-config_testFlipMemberVotingStates: journal open: applyTo=0 06:44:43,123 PM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-3-shard-cars-config_testFlipMemberVotingStates: Snapshot completed in 3.357 ms, resetting timer for the next recovery snapshot 06:44:43,124 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:44:43,124 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:44:43,124 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:43,125 PM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-3-shard-people-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:44:43,125 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 06:44:43,125 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testFlipMemberVotingStates 06:44:43,125 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testFlipMemberVotingStates 06:44:43,126 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@1ca3a869} 06:44:43,126 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:44:43,126 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testFlipMemberVotingStates 06:44:43,127 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Shard created, persistent : true 06:44:43,127 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#-1023395605] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:43,128 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 06:44:43,128 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:43,128 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testFlipMemberVotingStates is initialized 06:44:43,128 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:43,128 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from null to Follower 06:44:43,128 PM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-3-shard-cars-config_testFlipMemberVotingStates: Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 06:44:43,128 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 06:44:43,128 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from null to Follower 06:44:43,128 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates-notifier#-1989355339 created and ready for shard:member-3-shard-cars-oper_testFlipMemberVotingStates 06:44:43,129 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates-notifier#-1989355339] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:43,129 PM [cluster-test-shard-dispatcher-28] [INFO] RaftActor - member-3-shard-cars-oper_testFlipMemberVotingStates: Starting recovery with journal batch size 1 06:44:43,129 PM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-3-shard-people-config_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 06:44:43,129 PM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-3-shard-people-config_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 06:44:43,129 PM [cluster-test-shard-dispatcher-27] [INFO] Recovery - member-3-shard-people-config_testFlipMemberVotingStates: Taking snapshot on entry with index -1 06:44:43,130 PM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testFlipMemberVotingStates: journal open: applyTo=0 06:44:43,132 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:44:43,132 PM [cluster-test-shard-dispatcher-27] [INFO] Recovery - member-3-shard-people-config_testFlipMemberVotingStates: Snapshot completed in 2.776 ms, resetting timer for the next recovery snapshot 06:44:43,133 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:44:43,133 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:43,133 PM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-cars-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:44:43,127 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testFlipMemberVotingStates 06:44:43,134 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:44:43,134 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 06:44:43,134 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 06:44:43,134 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 06:44:43,134 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 06:44:43,135 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Shard created, persistent : true 06:44:43,135 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#1521474516] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:43,135 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates-notifier#-883498104 created and ready for shard:member-3-shard-people-oper_testFlipMemberVotingStates 06:44:43,136 PM [cluster-test-shard-dispatcher-29] [INFO] RaftActor - member-3-shard-people-oper_testFlipMemberVotingStates: Starting recovery with journal batch size 1 06:44:43,136 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates-notifier#-883498104] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:43,137 PM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testFlipMemberVotingStates: journal open: applyTo=0 06:44:43,138 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:44:43,139 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:44:43,139 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:43,139 PM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-people-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:44:43,140 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 06:44:43,140 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testFlipMemberVotingStates is initialized 06:44:43,140 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:43,140 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:43,140 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from null to Follower 06:44:43,140 PM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-3-shard-people-config_testFlipMemberVotingStates: Peer address for peer member-1-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 06:44:43,141 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 06:44:43,141 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from null to Follower 06:44:43,141 PM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-people-oper_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 06:44:43,141 PM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-cars-oper_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 06:44:43,142 PM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-people-oper_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 06:44:43,142 PM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-cars-oper_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 06:44:43,142 PM [cluster-test-shard-dispatcher-29] [INFO] Recovery - member-3-shard-people-oper_testFlipMemberVotingStates: Taking snapshot on entry with index -1 06:44:43,142 PM [cluster-test-shard-dispatcher-28] [INFO] Recovery - member-3-shard-cars-oper_testFlipMemberVotingStates: Taking snapshot on entry with index -1 06:44:43,145 PM [cluster-test-shard-dispatcher-29] [INFO] Recovery - member-3-shard-people-oper_testFlipMemberVotingStates: Snapshot completed in 3.377 ms, resetting timer for the next recovery snapshot 06:44:43,146 PM [cluster-test-shard-dispatcher-28] [INFO] Recovery - member-3-shard-cars-oper_testFlipMemberVotingStates: Snapshot completed in 3.926 ms, resetting timer for the next recovery snapshot 06:44:43,152 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 06:44:43,152 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testFlipMemberVotingStates is initialized 06:44:43,152 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:43,152 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:43,152 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from null to Follower 06:44:43,152 PM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-people-oper_testFlipMemberVotingStates: Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 06:44:43,152 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 06:44:43,152 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 06:44:43,152 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testFlipMemberVotingStates is initialized 06:44:43,152 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from null to Follower 06:44:43,153 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:43,153 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:43,153 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from null to Follower 06:44:43,153 PM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-3-shard-cars-oper_testFlipMemberVotingStates: Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 06:44:43,153 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 06:44:43,154 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from null to Follower 06:44:43,726 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 06:44:43,726 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 06:44:43,728 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:44:43,728 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 06:44:43,728 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:44:43,728 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 06:44:43,728 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 06:44:43,728 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 06:44:43,728 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 06:44:43,728 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 06:44:43,728 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 06:44:43,728 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 06:44:43,728 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:44:43,728 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 06:44:43,728 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:44:43,728 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 06:44:43,728 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 06:44:43,728 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 06:44:43,728 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 06:44:43,728 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 06:44:43,729 PM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-people-oper_testFlipMemberVotingStates: Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 06:44:43,728 PM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-1-shard-people-config_testFlipMemberVotingStates: Peer address for peer member-3-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 06:44:43,728 PM [cluster-test-shard-dispatcher-22] [INFO] PeerInfos - member-1-shard-cars-config_testFlipMemberVotingStates: Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 06:44:43,729 PM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-1-shard-people-config_testFlipMemberVotingStates: Peer address for peer member-2-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 06:44:43,728 PM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-1-shard-cars-oper_testFlipMemberVotingStates: Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 06:44:43,729 PM [cluster-test-shard-dispatcher-22] [INFO] PeerInfos - member-1-shard-cars-config_testFlipMemberVotingStates: Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 06:44:43,730 PM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-1-shard-cars-oper_testFlipMemberVotingStates: Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 06:44:43,728 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 06:44:43,730 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 06:44:43,730 PM [cluster-test-shard-dispatcher-24] [INFO] PeerInfos - member-1-shard-people-oper_testFlipMemberVotingStates: Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 06:44:43,744 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:44:43,744 PM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 06:44:43,744 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 06:44:43,744 PM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 06:44:43,744 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 06:44:43,744 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:44:43,744 PM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 06:44:43,744 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 06:44:43,744 PM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 06:44:43,744 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 06:44:43,744 PM [cluster-test-shard-dispatcher-35] [INFO] PeerInfos - member-2-shard-cars-config_testFlipMemberVotingStates: Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 06:44:43,744 PM [cluster-test-shard-dispatcher-35] [INFO] PeerInfos - member-2-shard-people-config_testFlipMemberVotingStates: Peer address for peer member-3-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 06:44:43,744 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:44:43,744 PM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 06:44:43,744 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 06:44:43,745 PM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 06:44:43,745 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 06:44:43,745 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:44:43,745 PM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 06:44:43,745 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 06:44:43,745 PM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 06:44:43,745 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 06:44:43,745 PM [cluster-test-shard-dispatcher-35] [INFO] PeerInfos - member-2-shard-cars-oper_testFlipMemberVotingStates: Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 06:44:43,745 PM [cluster-test-shard-dispatcher-36] [INFO] PeerInfos - member-2-shard-people-oper_testFlipMemberVotingStates: Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 06:44:43,891 PM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 2 06:44:43,892 PM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 06:44:43,892 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:43,892 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:43,892 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 06:44:43,892 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 06:44:43,898 PM [cluster-test-shard-dispatcher-34] [INFO] Follower - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 06:44:43,904 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:44:43,904 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 06:44:43,904 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 06:44:43,904 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 06:44:43,904 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 06:44:43,904 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:44:43,904 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 06:44:43,904 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 06:44:43,904 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:44:43,904 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 06:44:43,904 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 06:44:43,904 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 06:44:43,904 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 06:44:43,904 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 06:44:43,904 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 06:44:43,904 PM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-people-oper_testFlipMemberVotingStates: Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 06:44:43,905 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:44:43,905 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 06:44:43,905 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 06:44:43,905 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 06:44:43,905 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 06:44:43,905 PM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-3-shard-cars-config_testFlipMemberVotingStates: Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 06:44:43,905 PM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-3-shard-people-config_testFlipMemberVotingStates: Peer address for peer member-2-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 06:44:43,905 PM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-3-shard-cars-oper_testFlipMemberVotingStates: Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 06:44:43,907 PM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 06:44:43,910 PM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 06:44:43,910 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 06:44:43,910 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 06:44:43,910 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to PreLeader 06:44:43,911 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@607b4d44 06:44:43,911 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Candidate to PreLeader 06:44:43,914 PM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries{term=2, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 06:44:43,915 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7a2c5c6a 06:44:43,915 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done false 06:44:43,915 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done true 06:44:43,917 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@72fa60a9 06:44:43,917 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStates status sync done false 06:44:43,917 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStates status sync done true 06:44:43,922 PM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 06:44:43,923 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 06:44:43,923 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from PreLeader to Leader 06:44:43,923 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from PreLeader to Leader 06:44:43,927 PM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 2 06:44:43,927 PM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 06:44:43,927 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:43,927 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:43,927 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 06:44:43,927 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 06:44:43,929 PM [cluster-test-shard-dispatcher-36] [INFO] Follower - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 06:44:43,934 PM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 06:44:43,935 PM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 06:44:43,935 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 06:44:43,935 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 06:44:43,935 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to PreLeader 06:44:43,936 PM [cluster-test-shard-dispatcher-24] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 2 06:44:43,936 PM [cluster-test-shard-dispatcher-24] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 06:44:43,936 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:43,936 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:43,936 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6c244c2 06:44:43,936 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Candidate to PreLeader 06:44:43,937 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 06:44:43,937 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 06:44:43,938 PM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries{term=2, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 06:44:43,939 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done false 06:44:43,939 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@67e2cac2 06:44:43,939 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done true 06:44:43,940 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@10e3efc2 06:44:43,940 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStates status sync done false 06:44:43,940 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStates status sync done true 06:44:43,939 PM [cluster-test-shard-dispatcher-34] [INFO] Follower - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 06:44:43,946 PM [cluster-test-shard-dispatcher-22] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 06:44:43,946 PM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 06:44:43,946 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 06:44:43,946 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from PreLeader to Leader 06:44:43,946 PM [cluster-test-shard-dispatcher-22] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 06:44:43,946 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 06:44:43,946 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 06:44:43,946 PM [cluster-test-shard-dispatcher-24] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 2 06:44:43,947 PM [cluster-test-shard-dispatcher-24] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 06:44:43,947 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:43,947 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:43,947 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from PreLeader to Leader 06:44:43,947 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to PreLeader 06:44:43,947 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 06:44:43,947 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@223ed7ec 06:44:43,947 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 06:44:43,947 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Candidate to PreLeader 06:44:43,950 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@26dfc0f9 06:44:43,950 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 06:44:43,950 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done false 06:44:43,950 PM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 06:44:43,951 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done true 06:44:43,953 PM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries{term=2, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 06:44:43,955 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5901a72 06:44:43,955 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStates is ready 06:44:43,956 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStates status sync done false 06:44:43,956 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStates status sync done true 06:44:43,959 PM [cluster-test-shard-dispatcher-22] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 06:44:43,959 PM [cluster-test-shard-dispatcher-22] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 06:44:43,959 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 06:44:43,959 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 06:44:43,960 PM [cluster-test-shard-dispatcher-22] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 06:44:43,960 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 06:44:43,961 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to PreLeader 06:44:43,961 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from PreLeader to Leader 06:44:43,961 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from PreLeader to Leader 06:44:43,961 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 06:44:43,961 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@24e2423f 06:44:43,961 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} is now ready 06:44:43,961 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 06:44:43,961 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Candidate to PreLeader 06:44:43,962 PM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries{term=2, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 06:44:43,964 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done false 06:44:43,964 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@8ec1718 06:44:43,964 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 06:44:43,964 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@585d8d8e 06:44:43,964 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 06:44:43,965 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStates status sync done false 06:44:43,965 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStates status sync done true 06:44:43,965 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done true 06:44:43,972 PM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 06:44:43,973 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 06:44:43,973 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from PreLeader to Leader 06:44:43,973 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from PreLeader to Leader 06:44:43,973 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 06:44:43,973 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} is now ready 06:44:43,973 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 06:44:43,973 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} is now ready 06:44:43,973 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 06:44:43,973 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} is now ready 06:44:43,973 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#427315978] 06:44:43,974 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:44:43,974 PM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-737304395] 06:44:43,974 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-737304395] 06:44:43,977 PM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 06:44:43,977 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStates 06:44:43,978 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStates 06:44:43,978 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 06:44:43,979 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 06:44:43,980 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 06:44:43,980 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 06:44:43,980 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 06:44:43,980 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 06:44:43,981 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 06:44:43,981 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 06:44:43,986 PM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=false, member-2-shard-people-config_testFlipMemberVotingStates=false, member-3-shard-people-config_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 06:44:43,988 PM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=true, member-2-shard-cars-config_testFlipMemberVotingStates=false, member-1-shard-cars-config_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 06:44:43,988 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=true, member-1-shard-people-oper_testFlipMemberVotingStates=false, member-2-shard-people-oper_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 06:44:43,988 PM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=false, member-1-shard-cars-oper_testFlipMemberVotingStates=false, member-3-shard-cars-oper_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 06:44:44,8 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-oper_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 06:44:44,8 PM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 06:44:44,8 PM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard people 06:44:44,8 PM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 06:44:44,9 PM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 06:44:44,9 PM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 06:44:44,9 PM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 06:44:44,9 PM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 06:44:44,9 PM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 06:44:44,9 PM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 06:44:44,9 PM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard people 06:44:44,9 PM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 06:44:44,9 PM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 06:44:44,9 PM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 06:44:44,9 PM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard cars 06:44:44,9 PM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 06:44:44,9 PM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 06:44:44,9 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6e3b487e 06:44:44,9 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6e3b487e 06:44:44,9 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 06:44:44,10 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#427315978] 06:44:44,11 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@63951634 06:44:44,11 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:44:44,11 PM [cluster-test-shard-dispatcher-23] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 06:44:44,12 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4cabdb4f 06:44:44,12 PM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-737304395] 06:44:44,12 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4cabdb4f 06:44:44,12 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 06:44:44,12 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-737304395] 06:44:44,12 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@debc8af 06:44:44,12 PM [cluster-test-shard-dispatcher-22] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-config_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 06:44:44,12 PM [cluster-test-shard-dispatcher-24] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 06:44:44,12 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3c2bbc05 06:44:44,12 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3c2bbc05 06:44:44,12 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 06:44:44,13 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5ca39ee5 06:44:44,13 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1f57b54b 06:44:44,13 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@705a0a3 06:44:44,13 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@705a0a3 06:44:44,13 PM [cluster-test-shard-dispatcher-24] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 06:44:44,13 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#427315978] 06:44:44,13 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 06:44:44,13 PM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#-1957251928] 06:44:44,13 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#-1957251928] 06:44:44,13 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1162703038] 06:44:44,14 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 06:44:44,14 PM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#602092281] 06:44:44,14 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#602092281] 06:44:44,14 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1162703038] 06:44:44,14 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 06:44:44,14 PM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#1427234982] 06:44:44,14 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#1427234982] 06:44:44,15 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#75895194] 06:44:44,15 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 06:44:44,15 PM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#-932454036] 06:44:44,15 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#-932454036] 06:44:44,16 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#75895194] 06:44:44,16 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 06:44:44,16 PM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#-1889593918] 06:44:44,16 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#-1889593918] 06:44:44,17 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-1706716598] 06:44:44,17 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 06:44:44,17 PM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#-311609404] 06:44:44,18 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@40fc066 06:44:44,18 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1583c987 06:44:44,19 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#-311609404] 06:44:44,20 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-1706716598] 06:44:44,20 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 06:44:44,20 PM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#-1374956192] 06:44:44,20 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#-1374956192] 06:44:44,20 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#304900971] 06:44:44,20 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 06:44:44,20 PM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#-1807816411] 06:44:44,21 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 06:44:44,21 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@49f48392 06:44:44,21 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@473364dc 06:44:44,22 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 06:44:44,22 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#-1807816411] 06:44:44,22 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#304900971] 06:44:44,22 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 06:44:44,22 PM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#2068627754] 06:44:44,23 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#2068627754] 06:44:44,23 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#897167298] 06:44:44,23 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 06:44:44,23 PM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#-1023395605] 06:44:44,24 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#-1023395605] 06:44:44,25 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 06:44:44,26 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 06:44:44,27 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#897167298] 06:44:44,27 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 06:44:44,27 PM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#1521474516] 06:44:44,29 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#1521474516] 06:44:44,31 PM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-3-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 3 06:44:44,31 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-3-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 06:44:44,31 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:44,31 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:44,31 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 06:44:44,31 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@10cb0297 06:44:44,32 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 06:44:44,33 PM [cluster-test-shard-dispatcher-30] [INFO] Candidate - member-3-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 3 06:44:44,33 PM [cluster-test-shard-dispatcher-30] [INFO] RaftActorBehavior - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 06:44:44,33 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:44,33 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:44,33 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 06:44:44,33 PM [cluster-test-shard-dispatcher-30] [INFO] RaftActorBehavior - member-3-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 06:44:44,33 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 06:44:44,33 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 06:44:44,33 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 06:44:44,33 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@430ce8e2 06:44:44,33 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 06:44:44,33 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@60e6ee7 06:44:44,34 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from Candidate to Leader 06:44:44,35 PM [cluster-test-shard-dispatcher-29] [INFO] Candidate - member-3-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 3 06:44:44,35 PM [cluster-test-shard-dispatcher-29] [INFO] RaftActorBehavior - member-3-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 06:44:44,35 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:44,35 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:44,35 PM [cluster-test-shard-dispatcher-27] [INFO] Candidate - member-3-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 3 06:44:44,35 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 06:44:44,35 PM [cluster-test-shard-dispatcher-27] [INFO] RaftActorBehavior - member-3-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 06:44:44,35 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:44,35 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:44,35 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 06:44:44,35 PM [cluster-test-shard-dispatcher-29] [INFO] RaftActorBehavior - member-3-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 06:44:44,35 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 06:44:44,35 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 06:44:44,35 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to Leader 06:44:44,35 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1a9e2261 06:44:44,35 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 06:44:44,35 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7995745f 06:44:44,35 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 06:44:44,35 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2dcdd9f0 06:44:44,35 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from Candidate to Leader 06:44:44,35 PM [cluster-test-shard-dispatcher-27] [INFO] RaftActorBehavior - member-3-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 06:44:44,35 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 06:44:44,37 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 06:44:44,37 PM [cluster-test-shard-dispatcher-24] [INFO] AbstractLeader - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 2 - switching to Follower 06:44:44,38 PM [cluster-test-shard-dispatcher-36] [INFO] Follower - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 2 - updating term 06:44:44,38 PM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 2 - updating term 06:44:44,39 PM [cluster-test-shard-dispatcher-34] [INFO] Follower - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 2 - updating term 06:44:44,40 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 06:44:44,40 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6b8b95a7 06:44:44,40 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to Leader 06:44:44,40 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from Candidate to Leader 06:44:44,40 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStates is ready 06:44:44,40 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-3-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 06:44:44,40 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 06:44:44,40 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 06:44:44,40 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 06:44:44,40 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@70f6593 06:44:44,41 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from Candidate to Leader 06:44:44,41 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 06:44:44,41 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done false 06:44:44,41 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done false 06:44:44,41 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@225a547d 06:44:44,41 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@747fb586 06:44:44,41 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 06:44:44,41 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 06:44:44,42 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2f5117a2 06:44:44,42 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 06:44:44,42 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done false 06:44:44,42 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#427315978] 06:44:44,42 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:44:44,42 PM [cluster-test-shard-dispatcher-22] [INFO] AbstractLeader - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 2 - switching to Follower 06:44:44,42 PM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-737304395] 06:44:44,42 PM [cluster-test-shard-dispatcher-23] [INFO] AbstractLeader - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 2 - switching to Follower 06:44:44,42 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-737304395] 06:44:44,42 PM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 2 - updating term 06:44:44,43 PM [cluster-test-shard-dispatcher-24] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 06:44:44,43 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:44,43 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:44,43 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Leader to Follower 06:44:44,43 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@18a9ae53 06:44:44,43 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Leader to Follower 06:44:44,44 PM [cluster-test-shard-dispatcher-22] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 06:44:44,44 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:44,44 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:44,45 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Leader to Follower 06:44:44,45 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7fc3919e 06:44:44,45 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Leader to Follower 06:44:44,45 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done false 06:44:44,45 PM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 06:44:44,45 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:44,45 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:44,45 PM [cluster-test-shard-dispatcher-34] [INFO] RaftActorBehavior - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Cannot append entries because sender's term 2 is less than 3 06:44:44,45 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Leader to Follower 06:44:44,45 PM [cluster-test-shard-dispatcher-23] [INFO] AbstractLeader - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 2 - switching to Follower 06:44:44,46 PM [cluster-test-shard-dispatcher-34] [INFO] RaftActorBehavior - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Cannot append entries because sender's term 2 is less than 3 06:44:44,46 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@78193bce 06:44:44,46 PM [cluster-test-shard-dispatcher-29] [INFO] RaftActorBehavior - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Cannot append entries because sender's term 2 is less than 3 06:44:44,46 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Leader to Follower 06:44:44,47 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done true 06:44:44,48 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done true 06:44:44,48 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7e13d7f7 06:44:44,48 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 06:44:44,48 PM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 06:44:44,48 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:44,48 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:44,48 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Leader to Follower 06:44:44,48 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done true 06:44:44,49 PM [cluster-test-shard-dispatcher-34] [INFO] RaftActorBehavior - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Cannot append entries because sender's term 2 is less than 3 06:44:44,49 PM [cluster-test-shard-dispatcher-34] [INFO] RaftActorBehavior - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Cannot append entries because sender's term 2 is less than 3 06:44:44,50 PM [cluster-test-shard-dispatcher-34] [INFO] RaftActorBehavior - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Cannot append entries because sender's term 2 is less than 3 06:44:44,48 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@54b83423 06:44:44,50 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Leader to Follower 06:44:44,51 PM [cluster-test-shard-dispatcher-34] [INFO] RaftActorBehavior - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Cannot append entries because sender's term 2 is less than 3 06:44:44,52 PM [cluster-test-shard-dispatcher-34] [INFO] RaftActorBehavior - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Cannot append entries because sender's term 2 is less than 3 06:44:44,54 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done true 06:44:44,96 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-737304395] 06:44:44,147 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-737304395] 06:44:44,197 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-737304395] 06:44:44,248 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-737304395] 06:44:44,298 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-737304395] 06:44:44,349 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-737304395] 06:44:44,353 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: leader changed to member-3-shard-cars-oper_testFlipMemberVotingStates 06:44:44,353 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@46dbec58 06:44:44,353 PM [cluster-test-shard-dispatcher-22] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-cars-oper_testFlipMemberVotingStates in 341.7 ms 06:44:44,353 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: leader changed to member-3-shard-people-config_testFlipMemberVotingStates 06:44:44,354 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-cars-oper_testFlipMemberVotingStates status sync done false 06:44:44,354 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-people-config_testFlipMemberVotingStates in 341.3 ms 06:44:44,354 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: leader changed to member-3-shard-cars-config_testFlipMemberVotingStates 06:44:44,354 PM [cluster-test-shard-dispatcher-22] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-cars-config_testFlipMemberVotingStates in 341.3 ms 06:44:44,354 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@64f06d0 06:44:44,354 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@249006bf 06:44:44,354 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 06:44:44,354 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-people-config_testFlipMemberVotingStates status sync done false 06:44:44,354 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testFlipMemberVotingStates status sync done false 06:44:44,360 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: leader changed to member-3-shard-people-oper_testFlipMemberVotingStates 06:44:44,360 PM [cluster-test-shard-dispatcher-23] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-people-oper_testFlipMemberVotingStates in 350.9 ms 06:44:44,360 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@311c2722 06:44:44,360 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 06:44:44,360 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-people-oper_testFlipMemberVotingStates status sync done false 06:44:44,363 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testFlipMemberVotingStates status sync done true 06:44:44,363 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-cars-oper_testFlipMemberVotingStates status sync done true 06:44:44,363 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-people-config_testFlipMemberVotingStates status sync done true 06:44:44,364 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-people-oper_testFlipMemberVotingStates status sync done true 06:44:44,399 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-737304395] 06:44:44,400 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1162703038] 06:44:44,400 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 06:44:44,401 PM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#602092281] 06:44:44,401 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#602092281] 06:44:44,401 PM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 06:44:44,401 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStates 06:44:44,401 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStates 06:44:44,401 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 06:44:44,401 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 06:44:44,402 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 06:44:44,402 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 06:44:44,402 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 06:44:44,402 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 06:44:44,403 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 06:44:44,403 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 06:44:44,403 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=true, member-1-shard-cars-oper_testFlipMemberVotingStates=true, member-3-shard-cars-oper_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 06:44:44,404 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=false, member-1-shard-people-oper_testFlipMemberVotingStates=true, member-2-shard-people-oper_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 06:44:44,404 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=true, member-2-shard-people-config_testFlipMemberVotingStates=true, member-3-shard-people-config_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 06:44:44,404 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=false, member-2-shard-cars-config_testFlipMemberVotingStates=true, member-1-shard-cars-config_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 06:44:44,405 PM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-people-oper_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 06:44:44,405 PM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard people 06:44:44,405 PM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 06:44:44,405 PM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 06:44:44,405 PM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-people-config_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 06:44:44,405 PM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard people 06:44:44,405 PM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 06:44:44,405 PM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 06:44:44,405 PM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-cars-config_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 06:44:44,405 PM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 06:44:44,405 PM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 06:44:44,405 PM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 06:44:44,406 PM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-cars-oper_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 06:44:44,406 PM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard cars 06:44:44,406 PM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 06:44:44,406 PM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 06:44:44,406 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#427315978] 06:44:44,406 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:44:44,406 PM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-737304395] 06:44:44,406 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-737304395] 06:44:44,415 PM [cluster-test-shard-dispatcher-30] [INFO] RaftActorVotingConfigSupport - member-3-shard-people-oper_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 06:44:44,416 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2156b258 06:44:44,416 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3742098e 06:44:44,416 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-3-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3742098e 06:44:44,416 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 06:44:44,416 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorVotingConfigSupport - member-3-shard-cars-config_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 06:44:44,416 PM [cluster-test-shard-dispatcher-30] [INFO] RaftActorVotingConfigSupport - member-3-shard-people-config_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 06:44:44,416 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@681c141 06:44:44,416 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@132dac7c 06:44:44,416 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@d45efe7 06:44:44,416 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-3-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@132dac7c 06:44:44,416 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5b204ae4 06:44:44,416 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 06:44:44,416 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-3-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5b204ae4 06:44:44,416 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 06:44:44,418 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorVotingConfigSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 06:44:44,418 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 06:44:44,418 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5a666060 06:44:44,419 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@277f9fee 06:44:44,419 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@277f9fee 06:44:44,419 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 06:44:44,419 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 06:44:44,420 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 06:44:44,423 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2ec01b51 06:44:44,423 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6a89f2f1 06:44:44,423 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2b9aa118 06:44:44,426 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@77748e13 06:44:44,426 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7dc6a3f6 06:44:44,426 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@712837e3 06:44:44,428 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4ac40948 06:44:44,430 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2ef0c023 06:44:44,432 PM [cluster-test-shard-dispatcher-34] [INFO] Candidate - member-2-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 4 06:44:44,432 PM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-2-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 4 06:44:44,432 PM [cluster-test-shard-dispatcher-34] [INFO] RaftActorBehavior - member-2-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 06:44:44,432 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:44,433 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:44,433 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-2-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 06:44:44,433 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:44,433 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:44,433 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 06:44:44,433 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@11d7ac30 06:44:44,433 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 06:44:44,433 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 06:44:44,433 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2682e794 06:44:44,433 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 06:44:44,433 PM [cluster-test-shard-dispatcher-36] [INFO] Candidate - member-2-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 4 06:44:44,433 PM [cluster-test-shard-dispatcher-36] [INFO] RaftActorBehavior - member-2-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 06:44:44,433 PM [cluster-test-shard-dispatcher-36] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:44,433 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:44,434 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 06:44:44,434 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@48b0dec4 06:44:44,434 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 06:44:44,435 PM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote{term=4, candidateId=member-2-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than follower's term 3 - updating term 06:44:44,435 PM [cluster-test-shard-dispatcher-24] [INFO] Follower - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote{term=4, candidateId=member-2-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than follower's term 3 - updating term 06:44:44,435 PM [cluster-test-shard-dispatcher-22] [INFO] Follower - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote{term=4, candidateId=member-2-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than follower's term 3 - updating term 06:44:44,436 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 06:44:44,442 PM [cluster-test-shard-dispatcher-36] [INFO] Candidate - member-2-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 4 06:44:44,442 PM [cluster-test-shard-dispatcher-36] [INFO] RaftActorBehavior - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 06:44:44,442 PM [cluster-test-shard-dispatcher-36] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:44,442 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:44,442 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 06:44:44,442 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3b2f7d0a 06:44:44,442 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 06:44:44,443 PM [cluster-test-shard-dispatcher-22] [INFO] Follower - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote{term=4, candidateId=member-2-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than follower's term 3 - updating term 06:44:44,444 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-2-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 06:44:44,444 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 06:44:44,444 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 06:44:44,444 PM [cluster-test-shard-dispatcher-36] [INFO] RaftActorBehavior - member-2-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 06:44:44,444 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to Leader 06:44:44,444 PM [cluster-test-shard-dispatcher-36] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 06:44:44,444 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5ab2db27 06:44:44,444 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStates from Candidate to Leader 06:44:44,444 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@655f61b3 06:44:44,444 PM [cluster-test-shard-dispatcher-34] [INFO] RaftActorBehavior - member-2-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 06:44:44,444 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 06:44:44,444 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 06:44:44,444 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 06:44:44,444 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 06:44:44,444 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@723fe2ba 06:44:44,444 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to Leader 06:44:44,444 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStates from Candidate to Leader 06:44:44,444 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStates from Candidate to Leader 06:44:44,444 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 06:44:44,446 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@63dbad69 06:44:44,446 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testFlipMemberVotingStates status sync done false 06:44:44,446 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-people-config_testFlipMemberVotingStates status sync done false 06:44:44,446 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@36801745 06:44:44,446 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@413e793f 06:44:44,447 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-people-oper_testFlipMemberVotingStates status sync done false 06:44:44,447 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 06:44:44,447 PM [cluster-test-shard-dispatcher-32] [INFO] AbstractLeader - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntries{term=4, leaderId=member-2-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 3 - switching to Follower 06:44:44,447 PM [cluster-test-shard-dispatcher-12] [INFO] AbstractLeader - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntries{term=4, leaderId=member-2-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 3 - switching to Follower 06:44:44,447 PM [cluster-test-shard-dispatcher-30] [INFO] AbstractLeader - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntries{term=4, leaderId=member-2-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 3 - switching to Follower 06:44:44,450 PM [cluster-test-shard-dispatcher-32] [INFO] RaftActorBehavior - member-3-shard-cars-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 06:44:44,450 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:44,450 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:44,450 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from Leader to Follower 06:44:44,450 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-3-shard-people-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 06:44:44,450 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:44,450 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:44,450 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from Leader to Follower 06:44:44,450 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2ff2114b 06:44:44,450 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from Leader to Follower 06:44:44,450 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3ba2a29f 06:44:44,450 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from Leader to Follower 06:44:44,450 PM [cluster-test-shard-dispatcher-30] [INFO] RaftActorBehavior - member-3-shard-people-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 06:44:44,451 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:44,451 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:44,451 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from Leader to Follower 06:44:44,451 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@662191a8 06:44:44,451 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from Leader to Follower 06:44:44,451 PM [cluster-test-shard-dispatcher-36] [INFO] RaftActorBehavior - member-2-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 06:44:44,451 PM [cluster-test-shard-dispatcher-36] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 06:44:44,451 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 06:44:44,452 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testFlipMemberVotingStates status sync done true 06:44:44,452 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-people-config_testFlipMemberVotingStates status sync done true 06:44:44,452 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 06:44:44,452 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@48e89a58 06:44:44,452 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-people-oper_testFlipMemberVotingStates status sync done true 06:44:44,452 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStates from Candidate to Leader 06:44:44,452 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 06:44:44,454 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@18b49857 06:44:44,454 PM [cluster-test-shard-dispatcher-30] [INFO] AbstractLeader - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntries{term=4, leaderId=member-2-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 3 - switching to Follower 06:44:44,455 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 06:44:44,455 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-cars-oper_testFlipMemberVotingStates status sync done false 06:44:44,456 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-cars-oper_testFlipMemberVotingStates status sync done true 06:44:44,457 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-737304395] 06:44:44,457 PM [cluster-test-shard-dispatcher-30] [INFO] RaftActorBehavior - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 06:44:44,457 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:44,457 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:44,458 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from Leader to Follower 06:44:44,458 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@51300666 06:44:44,458 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#427315978] 06:44:44,458 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from Leader to Follower 06:44:44,458 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 06:44:44,458 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#-1957251928] 06:44:44,458 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#-1957251928] 06:44:44,458 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1162703038] 06:44:44,458 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 06:44:44,459 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#602092281] 06:44:44,459 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#602092281] 06:44:44,459 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1162703038] 06:44:44,459 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 06:44:44,459 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#1427234982] 06:44:44,459 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#1427234982] 06:44:44,459 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#75895194] 06:44:44,459 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 06:44:44,460 PM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#-932454036] 06:44:44,460 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#-932454036] 06:44:44,460 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#75895194] 06:44:44,460 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 06:44:44,460 PM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#-1889593918] 06:44:44,460 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#-1889593918] 06:44:44,461 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-1706716598] 06:44:44,461 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 06:44:44,461 PM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#-311609404] 06:44:44,461 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#-311609404] 06:44:44,461 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-1706716598] 06:44:44,461 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 06:44:44,461 PM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#-1374956192] 06:44:44,461 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#-1374956192] 06:44:44,462 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#304900971] 06:44:44,462 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 06:44:44,462 PM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#-1807816411] 06:44:44,462 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#-1807816411] 06:44:44,462 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#304900971] 06:44:44,462 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 06:44:44,462 PM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#2068627754] 06:44:44,462 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#2068627754] 06:44:44,462 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#897167298] 06:44:44,463 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 06:44:44,463 PM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#-1023395605] 06:44:44,463 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#-1023395605] 06:44:44,463 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#897167298] 06:44:44,463 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 06:44:44,464 PM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#1521474516] 06:44:44,464 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#1521474516] 06:44:44,464 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#427315978] 06:44:44,464 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:44:44,464 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-737304395] 06:44:44,465 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-737304395] 06:44:44,465 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 06:44:44,466 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testFlipMemberVotingStates 06:44:44,466 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testFlipMemberVotingStates 06:44:44,466 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 06:44:44,467 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Stopping Shard 06:44:44,467 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Stopping Shard 06:44:44,467 PM [cluster-test-shard-dispatcher-32] [INFO] EnabledRaftStorage - member-3-shard-people-config_testFlipMemberVotingStates: journal closed 06:44:44,468 PM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testFlipMemberVotingStates: journal closed 06:44:44,468 PM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 06:44:44,468 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 06:44:44,468 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 06:44:44,469 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testFlipMemberVotingStates 06:44:44,469 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testFlipMemberVotingStates 06:44:44,469 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 06:44:44,470 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Stopping Shard 06:44:44,470 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Stopping Shard 06:44:44,470 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testFlipMemberVotingStates: journal closed 06:44:44,470 PM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testFlipMemberVotingStates: journal closed 06:44:44,470 PM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 06:44:44,470 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 06:44:44,472 PM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:44:44,472 PM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:44:44,481 PM [cluster-test-pekko.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:44:44,483 PM [cluster-test-pekko.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:44:44,483 PM [cluster-test-pekko.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:44:44,484 PM [cluster-test-pekko.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:44:44,488 PM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 06:44:44,512 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 06:44:44,513 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testFlipMemberVotingStates 06:44:44,513 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testFlipMemberVotingStates 06:44:44,513 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 06:44:44,514 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6602b7f 06:44:44,514 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4945e704 06:44:44,514 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-2-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4945e704 06:44:44,514 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 06:44:44,514 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4667d194 06:44:44,514 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5f701681 06:44:44,514 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-2-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5f701681 06:44:44,514 PM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 06:44:44,518 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@100ccc59 06:44:44,518 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@dc08711 06:44:44,521 PM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-people-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 06:44:44,521 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 06:44:44,526 PM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 5 06:44:44,527 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 5 06:44:44,527 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:44,527 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:44,527 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 06:44:44,527 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3d8fffca 06:44:44,527 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 06:44:44,527 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 5 06:44:44,528 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 5 06:44:44,528 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:44,528 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:44,528 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 06:44:44,528 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5c97c9c5 06:44:44,528 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 06:44:44,528 PM [cluster-test-shard-dispatcher-12] [INFO] AbstractLeader - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Term 5 in "RequestVote{term=5, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than leader's term 4 - switching to Follower 06:44:44,529 PM [cluster-test-shard-dispatcher-34] [INFO] AbstractLeader - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Term 5 in "RequestVote{term=5, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than leader's term 4 - switching to Follower 06:44:44,535 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-2-shard-people-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 5 06:44:44,535 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:44,535 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:44,535 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , received role change from Leader to Follower 06:44:44,535 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@784a17ff 06:44:44,535 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStates from Leader to Follower 06:44:44,535 PM [cluster-test-shard-dispatcher-34] [INFO] RaftActorBehavior - member-2-shard-cars-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 5 06:44:44,535 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:44,535 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:44,535 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , received role change from Leader to Follower 06:44:44,535 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@c270f6a 06:44:44,535 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStates from Leader to Follower 06:44:44,537 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 06:44:44,537 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 06:44:44,537 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 06:44:44,537 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to Leader 06:44:44,537 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@51f6c44f 06:44:44,537 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Candidate to Leader 06:44:44,538 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 06:44:44,538 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 06:44:44,538 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 06:44:44,538 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@27eea42c 06:44:44,538 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to Leader 06:44:44,538 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Candidate to Leader 06:44:44,538 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 06:44:44,539 PM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-people-config_testFlipMemberVotingStates: leader changed to member-1-shard-people-config_testFlipMemberVotingStates 06:44:44,539 PM [cluster-test-shard-dispatcher-34] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-people-config_testFlipMemberVotingStates: Successfully transferred leadership to member-1-shard-people-config_testFlipMemberVotingStates in 25.88 ms 06:44:44,539 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testFlipMemberVotingStates: leader changed to member-1-shard-cars-config_testFlipMemberVotingStates 06:44:44,539 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testFlipMemberVotingStates: Successfully transferred leadership to member-1-shard-cars-config_testFlipMemberVotingStates in 25.67 ms 06:44:44,539 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done false 06:44:44,540 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4bc0c1d0 06:44:44,540 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Stopping Shard 06:44:44,540 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@47f6b8aa 06:44:44,540 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 06:44:44,540 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done false 06:44:44,540 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Stopping Shard 06:44:44,540 PM [cluster-test-shard-dispatcher-34] [INFO] EnabledRaftStorage - member-2-shard-people-config_testFlipMemberVotingStates: journal closed 06:44:44,541 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testFlipMemberVotingStates: journal closed 06:44:44,541 PM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 06:44:44,541 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 06:44:44,542 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 06:44:44,542 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testFlipMemberVotingStates 06:44:44,543 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testFlipMemberVotingStates 06:44:44,543 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 06:44:44,543 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@404a48f8 06:44:44,543 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@346d8032 06:44:44,543 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@346d8032 06:44:44,543 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 06:44:44,543 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@44b6c40b 06:44:44,543 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@38d9da14 06:44:44,543 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-2-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@44b6c40b 06:44:44,543 PM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 06:44:44,547 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2e9f57cf 06:44:44,547 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4ca47a38 06:44:44,548 PM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 06:44:44,548 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-people-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 06:44:44,553 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 5 06:44:44,553 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 5 06:44:44,553 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:44,553 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:44,553 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 06:44:44,553 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2b971b5f 06:44:44,553 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 06:44:44,554 PM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 5 06:44:44,554 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 5 06:44:44,554 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:44,554 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:44,554 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 06:44:44,554 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@43e30b02 06:44:44,554 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 06:44:44,555 PM [cluster-test-shard-dispatcher-12] [INFO] AbstractLeader - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Term 5 in "RequestVote{term=5, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than leader's term 4 - switching to Follower 06:44:44,556 PM [cluster-test-shard-dispatcher-34] [INFO] AbstractLeader - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Term 5 in "RequestVote{term=5, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than leader's term 4 - switching to Follower 06:44:44,562 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 5 06:44:44,562 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:44,562 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:44,562 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , received role change from Leader to Follower 06:44:44,562 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2a213df5 06:44:44,562 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStates from Leader to Follower 06:44:44,563 PM [cluster-test-shard-dispatcher-34] [INFO] RaftActorBehavior - member-2-shard-people-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 5 06:44:44,563 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 06:44:44,563 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:44,563 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , received role change from Leader to Follower 06:44:44,563 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@74117451 06:44:44,563 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStates from Leader to Follower 06:44:44,565 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 06:44:44,565 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 06:44:44,565 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 06:44:44,565 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 06:44:44,565 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@75ef1604 06:44:44,565 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Candidate to Leader 06:44:44,565 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 06:44:44,565 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 06:44:44,565 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 06:44:44,565 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 06:44:44,565 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@790a05c5 06:44:44,565 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Candidate to Leader 06:44:44,565 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 06:44:44,566 PM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-people-oper_testFlipMemberVotingStates: leader changed to member-1-shard-people-oper_testFlipMemberVotingStates 06:44:44,566 PM [cluster-test-shard-dispatcher-34] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-people-oper_testFlipMemberVotingStates: Successfully transferred leadership to member-1-shard-people-oper_testFlipMemberVotingStates in 23.37 ms 06:44:44,567 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done false 06:44:44,567 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Stopping Shard 06:44:44,568 PM [cluster-test-shard-dispatcher-34] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testFlipMemberVotingStates: journal closed 06:44:44,568 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-oper_testFlipMemberVotingStates: leader changed to member-1-shard-cars-oper_testFlipMemberVotingStates 06:44:44,568 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-cars-oper_testFlipMemberVotingStates: Successfully transferred leadership to member-1-shard-cars-oper_testFlipMemberVotingStates in 25.07 ms 06:44:44,568 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@75da529e 06:44:44,568 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done false 06:44:44,569 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Stopping Shard 06:44:44,569 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testFlipMemberVotingStates: journal closed 06:44:44,570 PM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 06:44:44,570 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 06:44:44,572 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:44:44,572 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:44:44,746 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 06:44:45,495 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [4363116322889847620] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 06:44:45,587 PM [cluster-test-pekko.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:44:45,587 PM [cluster-test-pekko.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:44:45,589 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/07/2025 18:44:45.593] [cluster-test-pekko.actor.internal-dispatcher-5] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:44:45,594 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-8425085304665297547] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 06:44:45,605 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 06:44:45,606 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testFlipMemberVotingStates 06:44:45,606 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testFlipMemberVotingStates 06:44:45,606 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 06:44:45,607 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2d7f1404 06:44:45,607 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4265ce29 06:44:45,607 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@49255ece 06:44:45,607 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1bb422eb 06:44:45,607 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@49255ece 06:44:45,607 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1bb422eb 06:44:45,607 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 06:44:45,607 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 06:44:46,694 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: leader transfer aborted 06:44:46,694 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: leader transfer aborted 06:44:46,694 PM [cluster-test-shard-dispatcher-12] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: Failed to transfer leadership in 1.087 s 06:44:46,694 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: In unpauseLeader 06:44:46,694 PM [cluster-test-shard-dispatcher-23] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: Failed to transfer leadership in 1.087 s 06:44:46,694 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: In unpauseLeader 06:44:46,695 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Stopping Shard 06:44:46,695 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Stopping Shard 06:44:46,696 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-people-config_testFlipMemberVotingStates: journal closed 06:44:46,696 PM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testFlipMemberVotingStates: journal closed 06:44:46,697 PM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 06:44:46,697 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 06:44:46,698 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 06:44:46,698 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testFlipMemberVotingStates 06:44:46,698 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testFlipMemberVotingStates 06:44:46,699 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 06:44:46,699 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@30cf3db3 06:44:46,699 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@b16898f 06:44:46,699 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@507628f4 06:44:46,699 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@30cf3db3 06:44:46,699 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 06:44:46,699 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@513b78d9 06:44:46,699 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@513b78d9 06:44:46,700 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 06:44:47,753 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: leader transfer aborted 06:44:47,753 PM [cluster-test-shard-dispatcher-22] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: Failed to transfer leadership in 1.055 s 06:44:47,754 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: In unpauseLeader 06:44:47,755 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Stopping Shard 06:44:47,755 PM [cluster-test-shard-dispatcher-22] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testFlipMemberVotingStates: journal closed 06:44:47,763 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: leader transfer aborted 06:44:47,763 PM [cluster-test-shard-dispatcher-22] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: Failed to transfer leadership in 1.065 s 06:44:47,763 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: In unpauseLeader 06:44:47,764 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Stopping Shard 06:44:47,764 PM [cluster-test-shard-dispatcher-22] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testFlipMemberVotingStates: journal closed 06:44:47,765 PM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 06:44:47,766 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 06:44:47,767 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:44:47,767 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:44:47,775 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/07/2025 18:44:47.778] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:44:47,803 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 06:44:47,826 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-7444145985044114527] 06:44:47,827 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 06:44:47,832 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 06:44:47,832 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 06:44:47,832 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:44:47,832 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:44:47,856 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:47,857 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 06:44:47,857 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 06:44:47,857 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 06:44:47,859 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:47,863 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testModuleShardLeaderMovement 06:44:47,868 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-1919890331] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:47,868 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 06:44:47,868 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 06:44:47,876 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 06:44:47,876 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testModuleShardLeaderMovement 06:44:47,877 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testModuleShardLeaderMovement 06:44:47,877 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:44:47,877 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 06:44:47,877 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 06:44:47,878 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 06:44:47,880 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@72663de6} 06:44:47,880 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} started 06:44:47,881 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:44:47,881 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testModuleShardLeaderMovement 06:44:47,882 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Shard created, persistent : true 06:44:47,883 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-config_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 06:44:47,884 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 06:44:47,884 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 06:44:47,884 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testModuleShardLeaderMovement on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement on shard member-1-shard-cars-config_testModuleShardLeaderMovement whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1431117530] 06:44:47,884 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testModuleShardLeaderMovement 06:44:47,884 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement-notifier#-2139872913 created and ready for shard:member-1-shard-cars-config_testModuleShardLeaderMovement 06:44:47,884 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1431117530] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:47,884 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement-notifier#-2139872913] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:47,884 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 06:44:47,884 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 06:44:47,884 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testModuleShardLeaderMovement on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement on shard member-1-shard-people-config_testModuleShardLeaderMovement whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement#-36114962] 06:44:47,884 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: Shard created, persistent : true 06:44:47,884 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement#-36114962] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:47,885 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement-notifier#614997077] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:47,885 PM [cluster-test-shard-dispatcher-14] [INFO] RaftActor - member-1-shard-people-config_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 06:44:47,885 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement-notifier#614997077 created and ready for shard:member-1-shard-people-config_testModuleShardLeaderMovement 06:44:47,886 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testModuleShardLeaderMovement: journal open: applyTo=0 06:44:47,886 PM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-1-shard-people-config_testModuleShardLeaderMovement: journal open: applyTo=0 06:44:47,887 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:47,887 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:47,887 PM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-1-shard-cars-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:44:47,887 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:44:47,890 PM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-1-shard-cars-config_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 06:44:47,891 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-config_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 06:44:47,891 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 06:44:47,891 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testModuleShardLeaderMovement is initialized 06:44:47,891 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 06:44:47,891 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:47,891 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from null to Follower 06:44:47,891 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 06:44:47,892 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testModuleShardLeaderMovement is initialized 06:44:47,892 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 06:44:47,892 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:47,892 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , received role change from null to Follower 06:44:47,894 PM [cluster-test-shard-dispatcher-14] [INFO] Candidate - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 1 06:44:47,894 PM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:44:47,894 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 06:44:47,894 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:47,894 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 06:44:47,894 PM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 06:44:47,894 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 06:44:47,895 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 06:44:47,895 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 06:44:47,895 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-config_testModuleShardLeaderMovement (Candidate): Starting new election term 1 06:44:47,895 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:44:47,895 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 06:44:47,895 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:47,895 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 06:44:47,895 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 06:44:47,895 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 06:44:47,896 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 06:44:47,896 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: became leader with frontend state for [] 06:44:47,896 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 06:44:47,896 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2a634302 06:44:47,896 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 06:44:47,899 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 06:44:47,899 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3d02e996 06:44:47,900 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testModuleShardLeaderMovement from Candidate to Leader 06:44:47,900 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 06:44:47,931 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-1919890331] 06:44:47,931 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:44:47,931 PM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1431117530] 06:44:47,932 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:47,937 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:47,942 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testModuleShardLeaderMovement 06:44:47,945 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-589001328] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:47,946 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 06:44:47,946 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 06:44:47,948 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 06:44:47,948 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testModuleShardLeaderMovement 06:44:47,948 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 06:44:47,948 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testModuleShardLeaderMovement 06:44:47,949 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:44:47,949 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 06:44:47,949 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 06:44:47,949 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@72a0d182} 06:44:47,949 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} started 06:44:47,950 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:44:47,950 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testModuleShardLeaderMovement 06:44:47,950 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 06:44:47,950 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 06:44:47,950 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testModuleShardLeaderMovement on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement on shard member-1-shard-cars-oper_testModuleShardLeaderMovement whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-469653173] 06:44:47,950 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testModuleShardLeaderMovement 06:44:47,951 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 06:44:47,951 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 06:44:47,951 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testModuleShardLeaderMovement on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement on shard member-1-shard-people-oper_testModuleShardLeaderMovement whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement#-30866001] 06:44:47,951 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Shard created, persistent : true 06:44:47,951 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-469653173] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:47,951 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: Shard created, persistent : true 06:44:47,951 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement#-30866001] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:47,952 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement-notifier#-607109403 created and ready for shard:member-1-shard-cars-oper_testModuleShardLeaderMovement 06:44:47,952 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement-notifier#-607109403] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:47,952 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-oper_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 06:44:47,952 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement-notifier#-1817046177 created and ready for shard:member-1-shard-people-oper_testModuleShardLeaderMovement 06:44:47,952 PM [cluster-test-shard-dispatcher-14] [INFO] RaftActor - member-1-shard-people-oper_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 06:44:47,952 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement-notifier#-1817046177] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:47,953 PM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testModuleShardLeaderMovement: journal open: applyTo=0 06:44:47,953 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testModuleShardLeaderMovement: journal open: applyTo=0 06:44:47,954 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:47,954 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:47,955 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:44:47,955 PM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-1-shard-cars-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:44:47,957 PM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-1-shard-cars-oper_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 06:44:47,958 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 06:44:47,958 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testModuleShardLeaderMovement is initialized 06:44:47,958 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 06:44:47,958 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:47,958 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from null to Follower 06:44:47,958 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-oper_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 06:44:47,959 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 06:44:47,959 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:47,959 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , received role change from null to Follower 06:44:47,960 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 06:44:47,960 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testModuleShardLeaderMovement is initialized 06:44:47,960 PM [cluster-test-shard-dispatcher-14] [INFO] Candidate - member-1-shard-cars-oper_testModuleShardLeaderMovement (Candidate): Starting new election term 1 06:44:47,961 PM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:44:47,961 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 06:44:47,961 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:47,961 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from Follower to Candidate 06:44:47,961 PM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 06:44:47,961 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 06:44:47,961 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: became leader with frontend state for [] 06:44:47,961 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from Candidate to Leader 06:44:47,961 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 06:44:47,961 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@395478a1 06:44:47,961 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-oper_testModuleShardLeaderMovement (Candidate): Starting new election term 1 06:44:47,961 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-oper_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:44:47,961 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testModuleShardLeaderMovement from Candidate to Leader 06:44:47,961 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 06:44:47,961 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:47,961 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , received role change from Follower to Candidate 06:44:47,961 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-oper_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 06:44:47,962 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 06:44:47,962 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: became leader with frontend state for [] 06:44:47,962 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , received role change from Candidate to Leader 06:44:47,962 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 06:44:47,962 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5985f36e 06:44:47,962 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testModuleShardLeaderMovement from Candidate to Leader 06:44:47,962 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 06:44:48,0 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-589001328] 06:44:48,0 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 06:44:48,0 PM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-469653173] 06:44:48,16 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 06:44:48,39 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [1936656591680954838] 06:44:48,40 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 06:44:48,43 PM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:44:48,43 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 06:44:48,45 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:44:48,47 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:44:48,56 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:48,59 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:48,62 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testModuleShardLeaderMovement 06:44:48,72 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1670117541] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:48,73 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 06:44:48,73 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 06:44:48,78 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 06:44:48,82 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 06:44:48,84 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@5b581427} 06:44:48,84 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} started 06:44:48,84 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:48,87 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:48,89 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:44:48,90 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testModuleShardLeaderMovement 06:44:48,93 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-1517027055] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:48,93 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 06:44:48,94 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 06:44:48,95 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 06:44:48,100 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 06:44:48,104 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@559fe13c} 06:44:48,107 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} started 06:44:48,110 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:44:48,132 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 06:44:48,145 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 06:44:48,150 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 06:44:48,150 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:44:48,150 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 06:44:48,151 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:44:48,151 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 06:44:48,171 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [8455932025584658676] 06:44:48,172 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 06:44:48,175 PM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:44:48,175 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 06:44:48,176 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:44:48,176 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:44:48,190 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:48,193 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:48,195 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testModuleShardLeaderMovement 06:44:48,199 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1743401531] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:48,199 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 06:44:48,200 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 06:44:48,206 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 06:44:48,207 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} started 06:44:48,208 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:48,210 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:44:48,214 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 06:44:48,215 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@4054fd0e} 06:44:48,215 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:44:48,217 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testModuleShardLeaderMovement 06:44:48,219 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-1267441137] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:48,219 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 06:44:48,219 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 06:44:48,220 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 06:44:48,226 PM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 06:44:48,227 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 06:44:48,228 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@4e930b74} 06:44:48,229 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} started 06:44:48,230 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:44:48,233 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:44:48,233 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:44:48,233 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 06:44:48,233 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 06:44:48,233 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 06:44:48,848 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 06:44:48,848 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 06:44:48,848 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:44:48,848 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:44:48,848 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement 06:44:48,848 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement 06:44:48,848 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 06:44:48,848 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 06:44:48,849 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-people-config_testModuleShardLeaderMovement 06:44:48,849 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-people-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 06:44:48,849 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 06:44:48,849 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:44:48,849 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement 06:44:48,849 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 06:44:48,849 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-people-config_testModuleShardLeaderMovement 06:44:48,849 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-people-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 06:44:48,849 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 06:44:48,849 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-people-oper_testModuleShardLeaderMovement 06:44:48,850 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-people-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 06:44:48,850 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 06:44:48,850 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:44:48,850 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement 06:44:48,850 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 06:44:48,850 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-people-oper_testModuleShardLeaderMovement 06:44:48,850 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-people-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 06:44:48,850 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 06:44:48,859 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:44:48,859 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:44:48,859 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 06:44:48,859 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 06:44:48,859 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:44:48,859 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 06:44:48,859 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:44:48,859 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 06:44:49,79 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:44:49,79 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 06:44:49,79 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:44:49,79 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 06:44:49,79 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:44:49,79 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 06:44:49,79 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:44:49,79 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 06:44:49,135 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 06:44:49,135 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 06:44:49,136 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 06:44:49,136 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement] 06:44:49,141 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 06:44:49,142 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1431117530, primaryVersion=13] 06:44:49,147 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1431117530 for shard member-2-shard-cars-config_testModuleShardLeaderMovement 06:44:49,148 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Shard created, persistent : true 06:44:49,148 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#716103225] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:49,149 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-notifier#351707930 created and ready for shard:member-2-shard-cars-config_testModuleShardLeaderMovement 06:44:49,149 PM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-2-shard-cars-config_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 06:44:49,149 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-notifier#351707930] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:49,150 PM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testModuleShardLeaderMovement: journal open: applyTo=0 06:44:49,150 PM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-config_testModuleShardLeaderMovement 06:44:49,151 PM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 06:44:49,151 PM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 06:44:49,151 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:49,151 PM [cluster-test-shard-dispatcher-13] [INFO] PekkoRecovery - member-2-shard-cars-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:44:49,155 PM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: snapshot is durable as of 2025-08-07T18:44:49.151132909Z 06:44:49,155 PM [cluster-test-shard-dispatcher-13] [INFO] PekkoRecovery - member-2-shard-cars-config_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 06:44:49,156 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 06:44:49,156 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testModuleShardLeaderMovement is initialized 06:44:49,156 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 06:44:49,156 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:49,156 PM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from null to Follower 06:44:49,156 PM [cluster-test-shard-dispatcher-13] [INFO] Follower - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 06:44:49,158 PM [cluster-test-shard-dispatcher-13] [INFO] Follower - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Snapshot received from leader: member-1-shard-cars-config_testModuleShardLeaderMovement 06:44:49,158 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 06:44:49,158 PM [cluster-test-shard-dispatcher-13] [INFO] SnapshotManager - member-2-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 06:44:49,159 PM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 06:44:49,159 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@60cfc6a4 06:44:49,159 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from null to Follower 06:44:49,159 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 06:44:49,162 PM [cluster-test-shard-dispatcher-13] [INFO] SnapshotManager - member-2-shard-cars-config_testModuleShardLeaderMovement: snapshot is durable as of 2025-08-07T18:44:49.159410875Z 06:44:49,162 PM [cluster-test-shard-dispatcher-13] [INFO] ShardSnapshotCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot 06:44:49,162 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 06:44:49,163 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-config_testModuleShardLeaderMovement: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#-2062720427] with name member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher 06:44:49,163 PM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#-2062720427] 06:44:49,163 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-2-shard-cars-config_testModuleShardLeaderMovement: state snapshot applied in 521.6 μs 06:44:49,163 PM [cluster-test-shard-dispatcher-13] [INFO] ShardSnapshotCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: Done applying snapshot 06:44:49,163 PM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testModuleShardLeaderMovement: publishChanges: rootPath: / 06:44:49,163 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#-2062720427] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:49,163 PM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher: 217.7 μs 06:44:49,165 PM [cluster-test-shard-dispatcher-14] [INFO] AbstractLeader - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-2-shard-cars-config_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to -1, nextIndex set to 0 06:44:49,168 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testModuleShardLeaderMovement} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1431117530 06:44:49,168 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard cars 06:44:49,169 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-config_testModuleShardLeaderMovement 06:44:49,169 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [cars] 06:44:49,171 PM [cluster-test-shard-dispatcher-14] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]} has been successfully replicated to a majority of followers 06:44:49,172 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 06:44:49,172 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:49,172 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6e0824cd 06:44:49,172 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 06:44:49,172 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 06:44:49,172 PM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 06:44:49,173 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 06:44:49,175 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1670117541] 06:44:49,175 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 06:44:49,175 PM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#716103225] 06:44:49,175 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#716103225] 06:44:49,176 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-1517027055] 06:44:49,176 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 06:44:49,176 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Local shard cars not found - shards present: [] 06:44:49,176 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 06:44:49,176 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 06:44:49,177 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 06:44:49,177 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement] 06:44:49,178 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 06:44:49,178 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-469653173, primaryVersion=13] 06:44:49,180 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-469653173 for shard member-2-shard-cars-oper_testModuleShardLeaderMovement 06:44:49,181 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Shard created, persistent : true 06:44:49,181 PM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement#-329068835] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:49,182 PM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-oper_testModuleShardLeaderMovement 06:44:49,182 PM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-notifier#-556443758 created and ready for shard:member-2-shard-cars-oper_testModuleShardLeaderMovement 06:44:49,182 PM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-notifier#-556443758] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:49,182 PM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-2-shard-cars-oper_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 06:44:49,182 PM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 06:44:49,182 PM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 06:44:49,185 PM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testModuleShardLeaderMovement: journal open: applyTo=0 06:44:49,186 PM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: snapshot is durable as of 2025-08-07T18:44:49.182740660Z 06:44:49,186 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:49,187 PM [cluster-test-shard-dispatcher-13] [INFO] PekkoRecovery - member-2-shard-cars-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:44:49,189 PM [cluster-test-shard-dispatcher-13] [INFO] PekkoRecovery - member-2-shard-cars-oper_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 06:44:49,189 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 06:44:49,189 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 06:44:49,189 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 06:44:49,189 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testModuleShardLeaderMovement is initialized 06:44:49,189 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:49,190 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testModuleShardLeaderMovement , received role change from null to Follower 06:44:49,190 PM [cluster-test-shard-dispatcher-13] [INFO] Follower - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 06:44:49,192 PM [cluster-test-shard-dispatcher-13] [INFO] Follower - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Snapshot received from leader: member-1-shard-cars-oper_testModuleShardLeaderMovement 06:44:49,192 PM [cluster-test-shard-dispatcher-13] [INFO] SnapshotManager - member-2-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 06:44:49,193 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done false 06:44:49,194 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 06:44:49,195 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2f352ef6 06:44:49,195 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testModuleShardLeaderMovement from null to Follower 06:44:49,195 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 06:44:49,195 PM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-2-shard-cars-oper_testModuleShardLeaderMovement: snapshot is durable as of 2025-08-07T18:44:49.193355132Z 06:44:49,195 PM [cluster-test-shard-dispatcher-11] [INFO] ShardSnapshotCohort - member-2-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot 06:44:49,195 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 06:44:49,196 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-oper_testModuleShardLeaderMovement: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-731267244] with name member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher 06:44:49,196 PM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-731267244] 06:44:49,196 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testModuleShardLeaderMovement: state snapshot applied in 441.5 μs 06:44:49,196 PM [cluster-test-shard-dispatcher-11] [INFO] ShardSnapshotCohort - member-2-shard-cars-oper_testModuleShardLeaderMovement: Done applying snapshot 06:44:49,196 PM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testModuleShardLeaderMovement: publishChanges: rootPath: / 06:44:49,196 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-731267244] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:49,196 PM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher: 81.50 μs 06:44:49,198 PM [cluster-test-shard-dispatcher-14] [INFO] AbstractLeader - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-2-shard-cars-oper_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to -1, nextIndex set to 0 06:44:49,200 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7fab85ae 06:44:49,201 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 06:44:49,201 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testModuleShardLeaderMovement} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-469653173 06:44:49,201 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard cars 06:44:49,201 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-oper_testModuleShardLeaderMovement 06:44:49,201 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars] 06:44:49,203 PM [cluster-test-shard-dispatcher-14] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]} has been successfully replicated to a majority of followers 06:44:49,204 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done true 06:44:49,204 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 06:44:49,204 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 06:44:49,204 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-1517027055] 06:44:49,204 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:49,204 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 06:44:49,204 PM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement#-329068835] 06:44:49,204 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement#-329068835] 06:44:49,205 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 06:44:49,205 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 06:44:49,206 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 06:44:49,206 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement, visitedAddresses: [pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement] 06:44:49,208 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 06:44:49,208 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1431117530, primaryVersion=13] 06:44:49,210 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1431117530 for shard member-3-shard-cars-config_testModuleShardLeaderMovement 06:44:49,210 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Shard created, persistent : true 06:44:49,211 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-1874919238] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:49,212 PM [cluster-test-shard-dispatcher-14] [INFO] RaftActor - member-3-shard-cars-config_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 06:44:49,213 PM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testModuleShardLeaderMovement: journal open: applyTo=0 06:44:49,214 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-notifier#-118534350 created and ready for shard:member-3-shard-cars-config_testModuleShardLeaderMovement 06:44:49,214 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-notifier#-118534350] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:49,214 PM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-config_testModuleShardLeaderMovement 06:44:49,215 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:49,215 PM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-3-shard-cars-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:44:49,215 PM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: Persising snapshot at EntryInfo[index=0, term=1]/EntryInfo[index=0, term=1] 06:44:49,215 PM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 06:44:49,218 PM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-3-shard-cars-config_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 06:44:49,218 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 06:44:49,218 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testModuleShardLeaderMovement is initialized 06:44:49,218 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 06:44:49,219 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:49,219 PM [cluster-test-shard-dispatcher-14] [INFO] Follower - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1], votingConfig=VotingConfig{serverInfo=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]}}" message is greater than follower's term 0 - updating term 06:44:49,219 PM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from null to Follower 06:44:49,219 PM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: snapshot is durable as of 2025-08-07T18:44:49.215806071Z 06:44:49,221 PM [cluster-test-shard-dispatcher-14] [INFO] Follower - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Snapshot received from leader: member-1-shard-cars-config_testModuleShardLeaderMovement 06:44:49,221 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 06:44:49,221 PM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-3-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 06:44:49,221 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@733daf50 06:44:49,221 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from null to Follower 06:44:49,221 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 06:44:49,221 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 06:44:49,224 PM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-3-shard-cars-config_testModuleShardLeaderMovement: snapshot is durable as of 2025-08-07T18:44:49.222109206Z 06:44:49,224 PM [cluster-test-shard-dispatcher-14] [INFO] ShardSnapshotCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot 06:44:49,224 PM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 06:44:49,224 PM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#-1920817961] 06:44:49,225 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-config_testModuleShardLeaderMovement: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#-1920817961] with name member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher 06:44:49,225 PM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testModuleShardLeaderMovement: publishChanges: rootPath: / 06:44:49,225 PM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#-1920817961] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:49,225 PM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher: 78.09 μs 06:44:49,225 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-3-shard-cars-config_testModuleShardLeaderMovement: state snapshot applied in 1.079 ms 06:44:49,225 PM [cluster-test-shard-dispatcher-14] [INFO] ShardSnapshotCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: Done applying snapshot 06:44:49,227 PM [cluster-test-shard-dispatcher-14] [INFO] AbstractLeader - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-3-shard-cars-config_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to 0, nextIndex set to 1 06:44:49,230 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testModuleShardLeaderMovement} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1431117530 06:44:49,230 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully added the replica shard cars 06:44:49,230 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-config_testModuleShardLeaderMovement 06:44:49,230 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [cars] 06:44:49,231 PM [cluster-test-shard-dispatcher-14] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: VotingConfig{serverInfo=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]} has been successfully replicated to a majority of followers 06:44:49,232 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 06:44:49,235 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 06:44:49,235 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:49,235 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7dfa4f47 06:44:49,235 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 06:44:49,235 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 06:44:49,235 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done true 06:44:49,235 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1743401531] 06:44:49,236 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 06:44:49,236 PM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-1874919238] 06:44:49,236 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-1874919238] 06:44:49,236 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-1267441137] 06:44:49,236 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 06:44:49,236 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: Local shard cars not found - shards present: [] 06:44:49,236 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 06:44:49,236 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 06:44:49,237 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 06:44:49,237 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement, visitedAddresses: [pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement] 06:44:49,239 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 06:44:49,239 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-469653173, primaryVersion=13] 06:44:49,241 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-469653173 for shard member-3-shard-cars-oper_testModuleShardLeaderMovement 06:44:49,241 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Shard created, persistent : true 06:44:49,242 PM [cluster-test-shard-dispatcher-14] [INFO] RaftActor - member-3-shard-cars-oper_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 06:44:49,244 PM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testModuleShardLeaderMovement: journal open: applyTo=0 06:44:49,244 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement#1425239968] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:49,244 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-notifier#1713034322] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:49,245 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-notifier#1713034322 created and ready for shard:member-3-shard-cars-oper_testModuleShardLeaderMovement 06:44:49,246 PM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-oper_testModuleShardLeaderMovement 06:44:49,247 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:44:49,247 PM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-3-shard-cars-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:44:49,247 PM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persising snapshot at EntryInfo[index=0, term=1]/EntryInfo[index=0, term=1] 06:44:49,247 PM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 06:44:49,250 PM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-3-shard-cars-oper_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 06:44:49,250 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 06:44:49,251 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:49,251 PM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: snapshot is durable as of 2025-08-07T18:44:49.247445772Z 06:44:49,251 PM [cluster-test-shard-dispatcher-14] [INFO] Follower - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1], votingConfig=VotingConfig{serverInfo=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]}}" message is greater than follower's term 0 - updating term 06:44:49,251 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 06:44:49,251 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testModuleShardLeaderMovement is initialized 06:44:49,251 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testModuleShardLeaderMovement , received role change from null to Follower 06:44:49,253 PM [cluster-test-shard-dispatcher-14] [INFO] Follower - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Snapshot received from leader: member-1-shard-cars-oper_testModuleShardLeaderMovement 06:44:49,253 PM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-3-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 06:44:49,253 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testModuleShardLeaderMovement status sync done false 06:44:49,254 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 06:44:49,254 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@79c5c420 06:44:49,254 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testModuleShardLeaderMovement from null to Follower 06:44:49,254 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 06:44:49,257 PM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-3-shard-cars-oper_testModuleShardLeaderMovement: snapshot is durable as of 2025-08-07T18:44:49.254697887Z 06:44:49,257 PM [cluster-test-shard-dispatcher-14] [INFO] ShardSnapshotCohort - member-3-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot 06:44:49,257 PM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendMetadata - member-3-shard-cars-oper_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 06:44:49,258 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-oper_testModuleShardLeaderMovement: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-441701584] with name member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher 06:44:49,258 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testModuleShardLeaderMovement: state snapshot applied in 530.3 μs 06:44:49,258 PM [cluster-test-shard-dispatcher-14] [INFO] ShardSnapshotCohort - member-3-shard-cars-oper_testModuleShardLeaderMovement: Done applying snapshot 06:44:49,258 PM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-441701584] 06:44:49,259 PM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-oper_testModuleShardLeaderMovement: publishChanges: rootPath: / 06:44:49,259 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-441701584] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:49,259 PM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-oper_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher: 144.3 μs 06:44:49,259 PM [cluster-test-shard-dispatcher-14] [INFO] AbstractLeader - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-3-shard-cars-oper_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to 0, nextIndex set to 1 06:44:49,262 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done false 06:44:49,262 PM [cluster-test-shard-dispatcher-14] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: VotingConfig{serverInfo=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]} has been successfully replicated to a majority of followers 06:44:49,263 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3cc7457c 06:44:49,263 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 06:44:49,263 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testModuleShardLeaderMovement} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-469653173 06:44:49,263 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully added the replica shard cars 06:44:49,263 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-oper_testModuleShardLeaderMovement 06:44:49,263 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [cars] 06:44:49,264 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done true 06:44:49,266 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 06:44:49,266 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:49,266 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testModuleShardLeaderMovement status sync done true 06:44:49,266 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 06:44:49,267 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-1267441137] 06:44:49,267 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 06:44:49,267 PM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement#1425239968] 06:44:49,267 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement#1425239968] 06:44:49,267 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-1919890331] 06:44:49,267 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:44:49,267 PM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1431117530] 06:44:49,267 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1431117530] 06:44:49,268 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1670117541] 06:44:49,268 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 06:44:49,268 PM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#716103225] 06:44:49,268 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#716103225] 06:44:49,268 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1743401531] 06:44:49,268 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 06:44:49,268 PM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-1874919238] 06:44:49,268 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-1874919238] 06:44:49,270 PM [main] [INFO] ClusterAdminRpcService - Moving leader to local node member-1 for shard ShardName{value=cars}, datastoreType Config 06:44:49,271 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-1919890331] 06:44:49,271 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:44:49,272 PM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1431117530] 06:44:49,272 PM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.MakeLeaderLocal to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1431117530] 06:44:49,274 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onMakeLeaderLocal received 06:44:49,274 PM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - Leadership transfer complete 06:44:49,275 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-1919890331] 06:44:49,275 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:44:49,275 PM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1431117530] 06:44:49,275 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1431117530] 06:44:49,279 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1670117541] 06:44:49,279 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 06:44:49,279 PM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#716103225] 06:44:49,279 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#716103225] 06:44:49,279 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1743401531] 06:44:49,279 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 06:44:49,279 PM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-1874919238] 06:44:49,279 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-1874919238] 06:44:49,280 PM [main] [INFO] ClusterAdminRpcService - Moving leader to local node member-2 for shard ShardName{value=cars}, datastoreType Config 06:44:49,280 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1670117541] 06:44:49,280 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 06:44:49,280 PM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#716103225] 06:44:49,280 PM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.MakeLeaderLocal to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#716103225] 06:44:49,280 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onMakeLeaderLocal received 06:44:49,293 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@12eff990 06:44:49,293 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7b5a3cf1 06:44:49,293 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@12eff990 06:44:49,293 PM [cluster-test-shard-dispatcher-14] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 06:44:49,295 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@229e2a9b 06:44:49,296 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 06:44:49,296 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6ca1d84d 06:44:49,304 PM [cluster-test-shard-dispatcher-14] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: leader transfer complete - waiting for new leader 06:44:49,306 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done true 06:44:49,310 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 2 06:44:49,310 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 2 06:44:49,310 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 06:44:49,310 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:49,310 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 06:44:49,310 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@43a6def6 06:44:49,310 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Follower to Candidate 06:44:49,311 PM [cluster-test-shard-dispatcher-14] [INFO] AbstractLeader - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Term 2 in "RequestVote{term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than leader's term 1 - switching to Follower 06:44:49,312 PM [cluster-test-shard-dispatcher-14] [INFO] Follower - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 2 in "RequestVote{term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than follower's term 1 - updating term 06:44:49,317 PM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to Follower, election term: 2 06:44:49,317 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 06:44:49,318 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:49,318 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to Follower 06:44:49,318 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@539a8a28 06:44:49,318 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Leader to Follower 06:44:49,319 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 2 06:44:49,319 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 06:44:49,319 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 06:44:49,319 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 06:44:49,319 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1cec52f7 06:44:49,319 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 06:44:49,319 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 06:44:49,320 PM [cluster-test-shard-dispatcher-14] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: leader changed to member-2-shard-cars-config_testModuleShardLeaderMovement 06:44:49,320 PM [cluster-test-shard-dispatcher-14] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: Successfully transferred leadership to member-2-shard-cars-config_testModuleShardLeaderMovement in 27.25 ms 06:44:49,320 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@759dddea 06:44:49,320 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 06:44:49,320 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done false 06:44:49,321 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 06:44:49,321 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@64da5a00 06:44:49,321 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 06:44:49,323 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done true 06:44:49,326 PM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ClusterAdminRpcService - Leadership transfer complete 06:44:49,326 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1670117541] 06:44:49,326 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 06:44:49,326 PM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#716103225] 06:44:49,326 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#716103225] 06:44:49,327 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done true 06:44:49,327 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-1919890331] 06:44:49,327 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:44:49,327 PM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1431117530] 06:44:49,327 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1431117530] 06:44:49,328 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1743401531] 06:44:49,328 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 06:44:49,328 PM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-1874919238] 06:44:49,328 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-1874919238] 06:44:49,328 PM [main] [INFO] ClusterAdminRpcService - Moving leader to local node member-3 for shard ShardName{value=cars}, datastoreType Config 06:44:49,328 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1743401531] 06:44:49,328 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 06:44:49,328 PM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-1874919238] 06:44:49,328 PM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.MakeLeaderLocal to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-1874919238] 06:44:49,328 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onMakeLeaderLocal received 06:44:49,330 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@36ad0b5 06:44:49,330 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@783ab078 06:44:49,330 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-2-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@783ab078 06:44:49,330 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 06:44:49,331 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@71a25781 06:44:49,332 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2e30171b 06:44:49,333 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: leader transfer complete - waiting for new leader 06:44:49,338 PM [cluster-test-shard-dispatcher-14] [INFO] Candidate - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 3 06:44:49,338 PM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 3 06:44:49,338 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 06:44:49,338 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:49,338 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 06:44:49,338 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@d374511 06:44:49,338 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from Follower to Candidate 06:44:49,339 PM [cluster-test-shard-dispatcher-11] [INFO] AbstractLeader - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Term 3 in "RequestVote{term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than leader's term 2 - switching to Follower 06:44:49,339 PM [cluster-test-shard-dispatcher-14] [INFO] Follower - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 3 in "RequestVote{term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than follower's term 2 - updating term 06:44:49,345 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to Follower, election term: 3 06:44:49,345 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 06:44:49,345 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:49,345 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to Follower 06:44:49,345 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1abb7c82 06:44:49,345 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Leader to Follower 06:44:49,345 PM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 06:44:49,345 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 06:44:49,346 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 06:44:49,346 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 06:44:49,346 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@264c2524 06:44:49,346 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 06:44:49,346 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 06:44:49,347 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@34a0d7d1 06:44:49,347 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 06:44:49,347 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done false 06:44:49,348 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: leader changed to member-3-shard-cars-config_testModuleShardLeaderMovement 06:44:49,348 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5a1d6054 06:44:49,348 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: Successfully transferred leadership to member-3-shard-cars-config_testModuleShardLeaderMovement in 18.14 ms 06:44:49,348 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 06:44:49,348 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 06:44:49,349 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done true 06:44:49,349 PM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - Leadership transfer complete 06:44:49,349 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1743401531] 06:44:49,350 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 06:44:49,350 PM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-1874919238] 06:44:49,350 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-1874919238] 06:44:49,350 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 06:44:49,351 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 06:44:49,351 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testModuleShardLeaderMovement 06:44:49,351 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 06:44:49,351 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@107071b7 06:44:49,351 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@16e72d41 06:44:49,351 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-3-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@16e72d41 06:44:49,351 PM [cluster-test-shard-dispatcher-26] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 06:44:49,353 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6f9f404d 06:44:49,353 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1e37d800 06:44:49,354 PM [cluster-test-shard-dispatcher-26] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: leader transfer complete - waiting for new leader 06:44:49,358 PM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 4 06:44:49,358 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 4 06:44:49,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 06:44:49,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:49,358 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 06:44:49,358 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5de24a1 06:44:49,359 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Follower to Candidate 06:44:49,359 PM [cluster-test-shard-dispatcher-26] [INFO] AbstractLeader - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than leader's term 3 - switching to Follower 06:44:49,359 PM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than follower's term 3 - updating term 06:44:49,364 PM [cluster-test-shard-dispatcher-26] [INFO] RaftActorBehavior - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to Follower, election term: 4 06:44:49,364 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 06:44:49,364 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:44:49,364 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to Follower 06:44:49,364 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1820d1ed 06:44:49,364 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from Leader to Follower 06:44:49,364 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 06:44:49,364 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 06:44:49,364 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 06:44:49,365 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 06:44:49,365 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@518c57de 06:44:49,365 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 06:44:49,365 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 06:44:49,366 PM [cluster-test-shard-dispatcher-26] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: leader changed to member-1-shard-cars-config_testModuleShardLeaderMovement 06:44:49,366 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3e56346d 06:44:49,366 PM [cluster-test-shard-dispatcher-26] [INFO] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: Successfully transferred leadership to member-1-shard-cars-config_testModuleShardLeaderMovement in 14.36 ms 06:44:49,366 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 06:44:49,366 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 06:44:49,366 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 06:44:49,366 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6cb3b912 06:44:49,366 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 06:44:49,366 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Stopping Shard 06:44:49,367 PM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testModuleShardLeaderMovement: journal closed 06:44:49,367 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 06:44:49,367 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 06:44:49,367 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 06:44:49,368 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda/0x00007f4ea8809e48] to Actor[pekko://cluster-test/user/$a#173477912] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/$a#173477912] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:49,368 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1743401531] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1743401531] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:49,368 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testModuleShardLeaderMovement 06:44:49,368 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 1 shards 06:44:49,369 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Stopping Shard 06:44:49,369 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 06:44:49,369 PM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testModuleShardLeaderMovement: journal closed 06:44:49,370 PM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 06:44:49,370 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 06:44:49,372 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:44:49,372 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:44:49,378 PM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:44:49,378 PM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:44:49,379 PM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:44:49,379 PM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:44:49,380 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 06:44:49,384 PM [cluster-test-pekko.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection closed with error: Connection reset 06:44:49,401 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 06:44:49,402 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testModuleShardLeaderMovement 06:44:49,402 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 1 shards 06:44:49,403 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Stopping Shard 06:44:49,404 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testModuleShardLeaderMovement: journal closed 06:44:49,404 PM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 06:44:49,405 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 06:44:49,405 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 06:44:49,405 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testModuleShardLeaderMovement 06:44:49,405 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 1 shards 06:44:49,406 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1670117541] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1670117541] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:49,406 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Stopping Shard 06:44:49,407 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda/0x00007f4ea8809e48] to Actor[pekko://cluster-test/user/$a#-987145174] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/$a#-987145174] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:44:49,407 PM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testModuleShardLeaderMovement: journal closed 06:44:49,407 PM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 06:44:49,408 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 06:44:49,409 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:44:49,409 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:44:49,867 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 06:44:50,386 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [8455932025584658676] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 06:44:50,425 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [1936656591680954838] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 06:44:50,430 PM [cluster-test-pekko.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:44:50,431 PM [cluster-test-pekko.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:44:50,433 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/07/2025 18:44:50.437] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:44:50,448 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 06:44:50,449 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testModuleShardLeaderMovement 06:44:50,449 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testModuleShardLeaderMovement 06:44:50,449 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 06:44:50,450 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@795ed368 06:44:50,450 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@63573d7b 06:44:50,450 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@795ed368 06:44:50,450 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@66a375c4 06:44:50,450 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@66a375c4 06:44:50,450 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 06:44:50,450 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: Stopping Shard 06:44:50,450 PM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-1-shard-people-config_testModuleShardLeaderMovement: journal closed 06:44:54,967 PM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Up)]. 06:44:54,967 PM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2559, Up)]. 06:44:54,967 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 06:44:54,968 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:44:54,968 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:44:54,968 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:44:54,968 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:44:56,861 PM [cluster-test-pekko.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 06:44:58,2 PM [cluster-test-pekko.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 06:44:59,595 PM [cluster-test-shard-dispatcher-14] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: leader transfer aborted 06:44:59,596 PM [cluster-test-shard-dispatcher-14] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: Failed to transfer leadership in 9.146 s 06:44:59,596 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: In unpauseLeader 06:44:59,596 PM [cluster-test-shard-dispatcher-14] [WARN] Leader - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-config_testModuleShardLeaderMovement from Leader to IsolatedLeader 06:44:59,597 PM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 4 06:44:59,597 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 06:44:59,597 PM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to IsolatedLeader 06:44:59,597 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Leader to IsolatedLeader 06:44:59,598 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Stopping Shard 06:44:59,598 PM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testModuleShardLeaderMovement: journal closed 06:44:59,599 PM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 06:44:59,599 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 06:44:59,600 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 06:44:59,600 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testModuleShardLeaderMovement 06:44:59,601 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testModuleShardLeaderMovement 06:44:59,601 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 06:44:59,601 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@57c4c1 06:44:59,601 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-oper_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@57c4c1 06:44:59,601 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@77993233 06:44:59,601 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: Stopping Shard 06:44:59,601 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3a54ef9a 06:44:59,601 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3a54ef9a 06:44:59,601 PM [cluster-test-shard-dispatcher-14] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 06:44:59,602 PM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testModuleShardLeaderMovement: journal closed 06:45:03,196 PM [cluster-test-shard-dispatcher-14] [WARN] Leader - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-oper_testModuleShardLeaderMovement from Leader to IsolatedLeader 06:45:03,196 PM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 06:45:03,196 PM [cluster-test-shard-dispatcher-14] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testModuleShardLeaderMovement: leader transfer aborted 06:45:03,197 PM [cluster-test-shard-dispatcher-14] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testModuleShardLeaderMovement: Failed to transfer leadership in 3.596 s 06:45:03,197 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: In unpauseLeader 06:45:03,197 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 06:45:03,197 PM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from Leader to IsolatedLeader 06:45:03,197 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testModuleShardLeaderMovement from Leader to IsolatedLeader 06:45:03,197 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Stopping Shard 06:45:03,198 PM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testModuleShardLeaderMovement: journal closed 06:45:03,198 PM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 06:45:03,199 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 06:45:03,200 PM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:45:03,200 PM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:45:03,205 PM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/07/2025 18:45:03.208] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:45:03,244 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 06:45:03,254 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [1127784267118406453] 06:45:03,254 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 06:45:03,257 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 06:45:03,257 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 06:45:03,261 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:45:03,261 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:45:03,272 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 06:45:03,272 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 06:45:03,272 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 06:45:03,273 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:03,275 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:03,277 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 06:45:03,282 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 06:45:03,283 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 06:45:03,288 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1001569632] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:03,291 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 06:45:03,291 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testRemoveShardReplica 06:45:03,292 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testRemoveShardReplica 06:45:03,292 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:03,292 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 06:45:03,292 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 06:45:03,298 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} 06:45:03,300 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@39c52514} 06:45:03,300 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} started 06:45:03,301 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:03,303 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:03,304 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:03,304 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testRemoveShardReplica 06:45:03,305 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica: Shard created, persistent : true 06:45:03,306 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-1858484956] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:03,306 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica-notifier#307596810 created and ready for shard:member-1-shard-cars-config_testRemoveShardReplica 06:45:03,306 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-1-shard-cars-config_testRemoveShardReplica: Starting recovery with journal batch size 1 06:45:03,307 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica-notifier#307596810] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:03,308 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 06:45:03,308 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testRemoveShardReplica: journal open: applyTo=0 06:45:03,308 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 06:45:03,308 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 06:45:03,310 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 06:45:03,310 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 06:45:03,311 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#1375420923] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:03,313 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica on shard member-1-shard-cars-config_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-1858484956] 06:45:03,313 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testRemoveShardReplica 06:45:03,314 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:03,314 PM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-1-shard-cars-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:03,315 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 06:45:03,315 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 06:45:03,315 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica on shard member-1-shard-people-config_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica#-695175802] 06:45:03,315 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica: Shard created, persistent : true 06:45:03,315 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica#-695175802] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:03,315 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-people-config_testRemoveShardReplica: Starting recovery with journal batch size 1 06:45:03,315 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica-notifier#-802246971] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:03,316 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica-notifier#-802246971 created and ready for shard:member-1-shard-people-config_testRemoveShardReplica 06:45:03,316 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-config_testRemoveShardReplica: journal open: applyTo=0 06:45:03,317 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:03,317 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:03,317 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardReplica, generation=0} 06:45:03,318 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardReplica, generation=0} started 06:45:03,319 PM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-1-shard-cars-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 06:45:03,319 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:03,319 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:03,319 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 06:45:03,320 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 06:45:03,320 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:03,320 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:03,320 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from null to Follower 06:45:03,320 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 06:45:03,320 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testRemoveShardReplica is initialized 06:45:03,320 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 06:45:03,320 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testRemoveShardReplica is initialized 06:45:03,320 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 06:45:03,320 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 06:45:03,320 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from null to Follower 06:45:03,324 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from null to Follower 06:45:03,324 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 06:45:03,324 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testRemoveShardReplica 06:45:03,325 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testRemoveShardReplica 06:45:03,325 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:03,325 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 06:45:03,325 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 06:45:03,325 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@7f8c496c} 06:45:03,326 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:03,326 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testRemoveShardReplica 06:45:03,326 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 06:45:03,326 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 06:45:03,326 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica on shard member-1-shard-cars-oper_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica#1280806407] 06:45:03,326 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testRemoveShardReplica 06:45:03,326 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 06:45:03,326 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 06:45:03,326 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica on shard member-1-shard-people-oper_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica#-1539618950] 06:45:03,327 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Shard created, persistent : true 06:45:03,327 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica#1280806407] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:03,327 PM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica-notifier#-1973682145 created and ready for shard:member-1-shard-cars-oper_testRemoveShardReplica 06:45:03,327 PM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica-notifier#-1973682145] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:03,328 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica#-1539618950] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:03,328 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica: Shard created, persistent : true 06:45:03,328 PM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica-notifier#-1558550125 created and ready for shard:member-1-shard-people-oper_testRemoveShardReplica 06:45:03,329 PM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica-notifier#-1558550125] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:03,329 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActor - member-1-shard-people-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 06:45:03,330 PM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testRemoveShardReplica: journal open: applyTo=0 06:45:03,331 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:03,331 PM [cluster-test-shard-dispatcher-19] [INFO] RaftActor - member-1-shard-cars-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 06:45:03,331 PM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:03,332 PM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testRemoveShardReplica: journal open: applyTo=0 06:45:03,333 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:03,333 PM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-1-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:03,335 PM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-people-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 06:45:03,336 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:03,336 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:03,336 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 06:45:03,336 PM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-1-shard-cars-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 06:45:03,336 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 06:45:03,336 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testRemoveShardReplica is initialized 06:45:03,336 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 06:45:03,336 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from null to Follower 06:45:03,337 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 06:45:03,337 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testRemoveShardReplica is initialized 06:45:03,338 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 06:45:03,338 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:03,338 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:03,339 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 06:45:03,339 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 06:45:03,339 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from null to Follower 06:45:03,352 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-882160981580280395] 06:45:03,353 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 06:45:03,357 PM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:45:03,357 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 06:45:03,372 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:45:03,372 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:45:03,379 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:03,382 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:03,384 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 06:45:03,398 PM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#77243579] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:03,398 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 06:45:03,398 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 06:45:03,410 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 06:45:03,410 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveShardReplica 06:45:03,411 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveShardReplica 06:45:03,413 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=0} 06:45:03,414 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@53a46ef1} 06:45:03,414 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=0} started 06:45:03,414 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:03,416 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:03,416 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:03,416 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveShardReplica 06:45:03,416 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveShardReplica 06:45:03,417 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica: Shard created, persistent : true 06:45:03,418 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 06:45:03,418 PM [cluster-test-shard-dispatcher-15] [INFO] RaftActor - member-2-shard-cars-config_testRemoveShardReplica: Starting recovery with journal batch size 1 06:45:03,419 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 06:45:03,419 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 06:45:03,419 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica: Shard created, persistent : true 06:45:03,419 PM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveShardReplica: journal open: applyTo=0 06:45:03,419 PM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-2-shard-people-config_testRemoveShardReplica: Starting recovery with journal batch size 1 06:45:03,420 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#1451480674 created and ready for shard:member-2-shard-people-config_testRemoveShardReplica 06:45:03,421 PM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveShardReplica: journal open: applyTo=0 06:45:03,421 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#1166684867] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:03,424 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:03,424 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#-1392348577] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:03,424 PM [cluster-test-shard-dispatcher-15] [INFO] PekkoRecovery - member-2-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:03,428 PM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica#896509494] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:03,428 PM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#1451480674] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:03,428 PM [cluster-test-shard-dispatcher-15] [INFO] PekkoRecovery - member-2-shard-people-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 06:45:03,428 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 06:45:03,428 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveShardReplica is initialized 06:45:03,429 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:03,429 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:03,429 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , received role change from null to Follower 06:45:03,429 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 06:45:03,429 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveShardReplica from null to Follower 06:45:03,430 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:03,430 PM [cluster-test-shard-dispatcher-15] [INFO] PekkoRecovery - member-2-shard-cars-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:03,432 PM [cluster-test-shard-dispatcher-15] [INFO] PekkoRecovery - member-2-shard-cars-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 06:45:03,433 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:03,433 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:03,433 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 06:45:03,433 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveShardReplica is initialized 06:45:03,437 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 06:45:03,437 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveShardReplica 06:45:03,437 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveShardReplica 06:45:03,439 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=0} 06:45:03,440 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@f829494} 06:45:03,440 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=0} started 06:45:03,440 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:03,440 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveShardReplica 06:45:03,441 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Shard created, persistent : true 06:45:03,442 PM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-2-shard-cars-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 06:45:03,443 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica#-1859843047] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:03,443 PM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveShardReplica: journal open: applyTo=0 06:45:03,443 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica-notifier#1483120806 created and ready for shard:member-2-shard-cars-oper_testRemoveShardReplica 06:45:03,443 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica-notifier#1483120806] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:03,444 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveShardReplica 06:45:03,444 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:03,445 PM [cluster-test-shard-dispatcher-13] [INFO] PekkoRecovery - member-2-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:03,446 PM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica#949215153] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:03,446 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica: Shard created, persistent : true 06:45:03,446 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica-notifier#1845818380 created and ready for shard:member-2-shard-people-oper_testRemoveShardReplica 06:45:03,447 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica-notifier#1845818380] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:03,447 PM [cluster-test-shard-dispatcher-25] [INFO] RaftActor - member-2-shard-people-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 06:45:03,447 PM [cluster-test-shard-dispatcher-13] [INFO] PekkoRecovery - member-2-shard-cars-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 06:45:03,448 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 06:45:03,448 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveShardReplica is initialized 06:45:03,448 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:03,448 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:03,448 PM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 06:45:03,448 PM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 06:45:03,448 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveShardReplica from null to Follower 06:45:03,449 PM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveShardReplica: journal open: applyTo=0 06:45:03,450 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:03,450 PM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-2-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:03,451 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#1672225799 created and ready for shard:member-2-shard-cars-config_testRemoveShardReplica 06:45:03,451 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 06:45:03,451 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 06:45:03,451 PM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#1672225799] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:03,451 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardReplica from null to Follower 06:45:03,452 PM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-2-shard-people-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 06:45:03,452 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 06:45:03,453 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveShardReplica is initialized 06:45:03,453 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:03,453 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:03,453 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 06:45:03,453 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 06:45:03,453 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveShardReplica from null to Follower 06:45:03,469 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 06:45:03,471 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 06:45:03,476 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:03,477 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 06:45:03,477 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 06:45:03,477 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 06:45:03,477 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 06:45:03,477 PM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveShardReplica: Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 06:45:03,477 PM [cluster-test-shard-dispatcher-13] [INFO] PeerInfos - member-2-shard-people-config_testRemoveShardReplica: Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 06:45:03,477 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:03,477 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 06:45:03,477 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 06:45:03,477 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 06:45:03,477 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 06:45:03,477 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 06:45:03,477 PM [cluster-test-shard-dispatcher-13] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 06:45:03,477 PM [cluster-test-shard-dispatcher-13] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveShardReplica: Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 06:45:03,496 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [6211997111722349875] 06:45:03,497 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 06:45:03,500 PM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:45:03,500 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 06:45:03,501 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:45:03,502 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:45:03,514 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:03,516 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:03,518 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 06:45:03,524 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1535798565] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:03,525 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 06:45:03,525 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 06:45:03,527 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 06:45:03,527 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testRemoveShardReplica 06:45:03,528 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testRemoveShardReplica 06:45:03,532 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} 06:45:03,533 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@79533db9} 06:45:03,534 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} started 06:45:03,534 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:03,534 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testRemoveShardReplica 06:45:03,534 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:03,536 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:03,539 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 06:45:03,540 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testRemoveShardReplica 06:45:03,540 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-3-shard-cars-config_testRemoveShardReplica: Shard created, persistent : true 06:45:03,540 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#1752421578] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:03,541 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica-notifier#-679824336 created and ready for shard:member-3-shard-cars-config_testRemoveShardReplica 06:45:03,541 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica-notifier#-679824336] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:03,541 PM [cluster-test-shard-dispatcher-22] [INFO] RaftActor - member-3-shard-cars-config_testRemoveShardReplica: Starting recovery with journal batch size 1 06:45:03,542 PM [cluster-test-shard-dispatcher-22] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testRemoveShardReplica: journal open: applyTo=0 06:45:03,543 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#-150606291] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:03,545 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:03,545 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-3-shard-cars-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:03,548 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 06:45:03,548 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 06:45:03,548 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardReplica, generation=0} 06:45:03,548 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica#-1853647662] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:03,548 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-3-shard-people-config_testRemoveShardReplica: Shard created, persistent : true 06:45:03,549 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 06:45:03,549 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testRemoveShardReplica 06:45:03,549 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-3-shard-cars-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 06:45:03,549 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testRemoveShardReplica 06:45:03,549 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@2da4dae4} 06:45:03,549 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 06:45:03,549 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testRemoveShardReplica is initialized 06:45:03,550 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:03,550 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:03,550 PM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 06:45:03,550 PM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 06:45:03,550 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardReplica from null to Follower 06:45:03,550 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardReplica, generation=0} started 06:45:03,551 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica-notifier#1838788505 created and ready for shard:member-3-shard-people-config_testRemoveShardReplica 06:45:03,552 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} 06:45:03,552 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:03,552 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testRemoveShardReplica 06:45:03,553 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testRemoveShardReplica 06:45:03,553 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardReplica: Shard created, persistent : true 06:45:03,553 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-3-shard-people-oper_testRemoveShardReplica: Shard created, persistent : true 06:45:03,554 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica-notifier#-1461412762 created and ready for shard:member-3-shard-cars-oper_testRemoveShardReplica 06:45:03,554 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica-notifier#1904255183 created and ready for shard:member-3-shard-people-oper_testRemoveShardReplica 06:45:03,551 PM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica-notifier#1838788505] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:03,554 PM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica#1287344666] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:03,554 PM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica#-1920631366] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:03,554 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-3-shard-cars-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 06:45:03,554 PM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica-notifier#-1461412762] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:03,554 PM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica-notifier#1904255183] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:03,561 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testRemoveShardReplica: journal open: applyTo=0 06:45:03,563 PM [cluster-test-shard-dispatcher-22] [INFO] RaftActor - member-3-shard-people-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 06:45:03,564 PM [cluster-test-shard-dispatcher-22] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testRemoveShardReplica: journal open: applyTo=0 06:45:03,566 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:03,566 PM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:03,567 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:03,567 PM [cluster-test-shard-dispatcher-22] [INFO] PekkoRecovery - member-3-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:03,568 PM [cluster-test-shard-dispatcher-24] [INFO] RaftActor - member-3-shard-people-config_testRemoveShardReplica: Starting recovery with journal batch size 1 06:45:03,569 PM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-cars-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 06:45:03,569 PM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-3-shard-people-config_testRemoveShardReplica: journal open: applyTo=0 06:45:03,569 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 06:45:03,569 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testRemoveShardReplica is initialized 06:45:03,570 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:03,570 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:03,570 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 06:45:03,570 PM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 06:45:03,570 PM [cluster-test-shard-dispatcher-22] [INFO] PekkoRecovery - member-3-shard-people-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 06:45:03,570 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:03,570 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:03,570 PM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 06:45:03,570 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testRemoveShardReplica from null to Follower 06:45:03,570 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 06:45:03,570 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testRemoveShardReplica is initialized 06:45:03,570 PM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 06:45:03,570 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:03,570 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testRemoveShardReplica from null to Follower 06:45:03,571 PM [cluster-test-shard-dispatcher-24] [INFO] PekkoRecovery - member-3-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:03,573 PM [cluster-test-shard-dispatcher-24] [INFO] PekkoRecovery - member-3-shard-people-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 06:45:03,574 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 06:45:03,574 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testRemoveShardReplica is initialized 06:45:03,574 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:03,574 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:03,574 PM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardReplica , received role change from null to Follower 06:45:03,574 PM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 06:45:03,574 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testRemoveShardReplica from null to Follower 06:45:03,581 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 06:45:03,583 PM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 06:45:03,585 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:03,585 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 06:45:03,585 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 06:45:03,585 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 06:45:03,585 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 06:45:03,585 PM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-3-shard-people-oper_testRemoveShardReplica: Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 06:45:03,585 PM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-3-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 06:45:03,585 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:03,585 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 06:45:03,585 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 06:45:03,586 PM [cluster-test-shard-dispatcher-22] [INFO] PeerInfos - member-3-shard-cars-config_testRemoveShardReplica: Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 06:45:03,586 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 06:45:03,586 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 06:45:03,587 PM [cluster-test-shard-dispatcher-22] [INFO] PeerInfos - member-3-shard-people-config_testRemoveShardReplica: Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 06:45:03,673 PM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-people-config_testRemoveShardReplica (Candidate): Starting new election term 1 06:45:03,673 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:45:03,673 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:03,673 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:03,673 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from Follower to Candidate 06:45:03,673 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from Follower to Candidate 06:45:03,684 PM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Starting new election term 1 06:45:03,684 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:45:03,684 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:03,684 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:03,684 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from Follower to Candidate 06:45:03,684 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from Follower to Candidate 06:45:03,684 PM [cluster-test-shard-dispatcher-19] [INFO] Candidate - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Starting new election term 1 06:45:03,685 PM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:45:03,685 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:03,685 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:03,685 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from Follower to Candidate 06:45:03,685 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from Follower to Candidate 06:45:03,734 PM [cluster-test-shard-dispatcher-19] [INFO] Candidate - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Starting new election term 1 06:45:03,734 PM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:45:03,734 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:03,734 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:03,734 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from Follower to Candidate 06:45:03,734 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from Follower to Candidate 06:45:04,4 PM [cluster-test-shard-dispatcher-19] [INFO] Candidate - member-1-shard-people-config_testRemoveShardReplica (Candidate): Starting new election term 2 06:45:04,13 PM [cluster-test-shard-dispatcher-19] [INFO] Candidate - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Starting new election term 2 06:45:04,53 PM [cluster-test-shard-dispatcher-19] [INFO] Candidate - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Starting new election term 2 06:45:04,73 PM [cluster-test-shard-dispatcher-19] [INFO] Candidate - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Starting new election term 2 06:45:04,274 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 06:45:04,274 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:04,274 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 06:45:04,274 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 06:45:04,274 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 06:45:04,275 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 06:45:04,275 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 06:45:04,275 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:04,275 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 06:45:04,275 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 06:45:04,275 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 06:45:04,275 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 06:45:04,275 PM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-1-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-3-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 06:45:04,275 PM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-1-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-2-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 06:45:04,275 PM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-1-shard-people-oper_testRemoveShardReplica: Peer address for peer member-3-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 06:45:04,275 PM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-1-shard-people-oper_testRemoveShardReplica: Peer address for peer member-2-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 06:45:04,274 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:04,276 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 06:45:04,276 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 06:45:04,276 PM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-1-shard-cars-config_testRemoveShardReplica: Peer address for peer member-3-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 06:45:04,276 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 06:45:04,276 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 06:45:04,276 PM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-1-shard-people-config_testRemoveShardReplica: Peer address for peer member-3-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 06:45:04,276 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:04,276 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 06:45:04,276 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 06:45:04,276 PM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-1-shard-cars-config_testRemoveShardReplica: Peer address for peer member-2-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 06:45:04,277 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 06:45:04,277 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 06:45:04,277 PM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-1-shard-people-config_testRemoveShardReplica: Peer address for peer member-2-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 06:45:04,283 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:04,283 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 06:45:04,283 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 06:45:04,283 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 06:45:04,283 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 06:45:04,283 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:04,283 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 06:45:04,283 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 06:45:04,283 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 06:45:04,283 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 06:45:04,283 PM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveShardReplica: Peer address for peer member-3-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 06:45:04,283 PM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-2-shard-people-config_testRemoveShardReplica: Peer address for peer member-3-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 06:45:04,283 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:04,283 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 06:45:04,283 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 06:45:04,283 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 06:45:04,283 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 06:45:04,283 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:04,283 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 06:45:04,283 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 06:45:04,283 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 06:45:04,283 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 06:45:04,283 PM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-3-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 06:45:04,283 PM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveShardReplica: Peer address for peer member-3-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 06:45:04,346 PM [cluster-test-shard-dispatcher-19] [INFO] Candidate - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Starting new election term 3 06:45:04,350 PM [cluster-test-shard-dispatcher-25] [INFO] Follower - member-2-shard-people-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:04,350 PM [cluster-test-shard-dispatcher-24] [INFO] Follower - member-3-shard-people-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:04,359 PM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 06:45:04,359 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 06:45:04,360 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: became leader with frontend state for [] 06:45:04,360 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from Candidate to Leader 06:45:04,360 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardReplica status sync done false 06:45:04,361 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1942ec82 06:45:04,361 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7fdde6c5 06:45:04,361 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveShardReplica status sync done false 06:45:04,361 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@47227127 06:45:04,361 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from Candidate to Leader 06:45:04,364 PM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-people-config_testRemoveShardReplica (Candidate): Starting new election term 3 06:45:04,365 PM [cluster-test-shard-dispatcher-22] [INFO] Follower - member-3-shard-people-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:04,366 PM [cluster-test-shard-dispatcher-31] [INFO] Follower - member-2-shard-people-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:04,373 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 06:45:04,373 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 06:45:04,373 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: became leader with frontend state for [] 06:45:04,373 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from Candidate to Leader 06:45:04,374 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3f7f1db3 06:45:04,374 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from Candidate to Leader 06:45:04,375 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1a6daa6b 06:45:04,375 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveShardReplica status sync done false 06:45:04,376 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardReplica status sync done false 06:45:04,376 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7c04ce4 06:45:04,403 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:04,403 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 06:45:04,403 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 06:45:04,403 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 06:45:04,403 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 06:45:04,403 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:04,403 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 06:45:04,403 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 06:45:04,403 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 06:45:04,403 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 06:45:04,403 PM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-3-shard-cars-config_testRemoveShardReplica: Peer address for peer member-2-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 06:45:04,403 PM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-3-shard-people-config_testRemoveShardReplica: Peer address for peer member-2-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 06:45:04,403 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:04,403 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 06:45:04,403 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 06:45:04,403 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 06:45:04,403 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 06:45:04,404 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:04,404 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 06:45:04,404 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 06:45:04,404 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 06:45:04,404 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 06:45:04,404 PM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-3-shard-people-oper_testRemoveShardReplica: Peer address for peer member-2-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 06:45:04,404 PM [cluster-test-shard-dispatcher-34] [INFO] PeerInfos - member-3-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-2-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 06:45:04,413 PM [cluster-test-shard-dispatcher-19] [INFO] Candidate - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Starting new election term 3 06:45:04,415 PM [cluster-test-shard-dispatcher-25] [INFO] Follower - member-2-shard-cars-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:04,415 PM [cluster-test-shard-dispatcher-34] [INFO] Follower - member-3-shard-cars-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:04,424 PM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 06:45:04,424 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 06:45:04,424 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: became leader with frontend state for [] 06:45:04,424 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from Candidate to Leader 06:45:04,424 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3c68cfe3 06:45:04,425 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from Candidate to Leader 06:45:04,425 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardReplica is ready 06:45:04,425 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} is now ready 06:45:04,425 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} 06:45:04,426 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5efd6ff4 06:45:04,427 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardReplica is ready 06:45:04,427 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done false 06:45:04,433 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@62551149 06:45:04,433 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardReplica is ready 06:45:04,433 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} is now ready 06:45:04,433 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1001569632] 06:45:04,434 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:45:04,434 PM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-1858484956] 06:45:04,434 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-1858484956] 06:45:04,434 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardReplica status sync done false 06:45:04,434 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#77243579] 06:45:04,434 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 06:45:04,434 PM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#1166684867] 06:45:04,434 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#1166684867] 06:45:04,435 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1535798565] 06:45:04,435 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 06:45:04,435 PM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#1752421578] 06:45:04,435 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#1752421578] 06:45:04,437 PM [main] [INFO] ClusterAdminRpcService - Removing replica for shard ShardName{value=cars} memberName MemberName{value=member-3}, datastoreType Config 06:45:04,437 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-3}] 06:45:04,439 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 06:45:04,439 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica, primaryVersion=13] 06:45:04,440 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica for shard member-3-shard-cars-config_testRemoveShardReplica 06:45:04,447 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveShardReplica} for shard cars 06:45:04,447 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard cars 06:45:04,447 PM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] ClusterAdminRpcService - Successfully removed replica for shard ShardName{value=cars} 06:45:04,448 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1001569632] 06:45:04,448 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:45:04,448 PM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-1858484956] 06:45:04,448 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-1858484956] 06:45:04,449 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#77243579] 06:45:04,449 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 06:45:04,449 PM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#1166684867] 06:45:04,449 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#1166684867] 06:45:04,451 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testRemoveShardReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]} has been successfully replicated to a majority of followers 06:45:04,452 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done true 06:45:04,452 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1535798565] 06:45:04,452 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 06:45:04,452 PM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#1752421578] 06:45:04,454 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#1752421578] with 27000 ms timeout 06:45:04,454 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-config_testRemoveShardReplica: Stopping Shard 06:45:04,455 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testRemoveShardReplica: journal closed 06:45:04,455 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard cars has been removed 06:45:04,455 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [people] 06:45:04,455 PM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#1752421578] 06:45:04,473 PM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Starting new election term 3 06:45:04,475 PM [cluster-test-shard-dispatcher-25] [INFO] Follower - member-2-shard-cars-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:04,475 PM [cluster-test-shard-dispatcher-34] [INFO] Follower - member-3-shard-cars-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:04,482 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 06:45:04,483 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 06:45:04,483 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: became leader with frontend state for [] 06:45:04,483 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from Candidate to Leader 06:45:04,483 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@494219a3 06:45:04,483 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from Candidate to Leader 06:45:04,483 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardReplica is ready 06:45:04,484 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardReplica status sync done false 06:45:04,484 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@691021d8 06:45:04,484 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardReplica is ready 06:45:04,484 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2876e0d4 06:45:04,485 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testRemoveShardReplica is ready 06:45:04,485 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveShardReplica status sync done false 06:45:04,502 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1535798565] 06:45:04,502 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 06:45:04,502 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard cars not found - shards present: [people] 06:45:04,503 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=0} 06:45:04,503 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testRemoveShardReplica 06:45:04,504 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveShardReplica 06:45:04,504 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 06:45:04,504 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node [pekko://cluster-test@127.0.0.1:2559] as [Down] 06:45:04,508 PM [cluster-test-pekko.actor.default-dispatcher-29] [WARN] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Received gossip where this member has been downed, from [pekko://cluster-test@127.0.0.1:2558] 06:45:04,509 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica: Stopping Shard 06:45:04,509 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica: Stopping Shard 06:45:04,510 PM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveShardReplica: journal closed 06:45:04,510 PM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveShardReplica: journal closed 06:45:04,510 PM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Node has been marked as DOWN. Shutting down myself 06:45:04,510 PM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 06:45:04,510 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 06:45:04,511 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=0} 06:45:04,511 PM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Shutting down... 06:45:04,511 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Successfully shut down 06:45:04,511 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveShardReplica 06:45:04,511 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveShardReplica 06:45:04,511 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 06:45:04,512 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Stopping Shard 06:45:04,512 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica: Stopping Shard 06:45:04,513 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberRemoved: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:04,514 PM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveShardReplica: journal closed 06:45:04,515 PM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveShardReplica: journal closed 06:45:04,515 PM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 06:45:04,515 PM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 06:45:04,516 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberRemoved: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:04,516 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberRemoved: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:04,516 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 06:45:04,518 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:45:04,518 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] CoordinatedShutdown - Running CoordinatedShutdown with reason [ClusterDowningReason] 06:45:04,518 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:45:04,526 PM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:04,526 PM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:04,526 PM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:04,526 PM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:04,528 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 06:45:04,555 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 06:45:04,566 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-303998725495193994] 06:45:04,566 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 06:45:04,567 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Registered cluster JMX MBean [pekko:type=Cluster] 06:45:04,567 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 06:45:04,568 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:45:04,568 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:45:04,578 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:04,582 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:04,585 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 06:45:04,593 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#1749668930] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:04,593 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 06:45:04,594 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 06:45:04,599 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 06:45:04,599 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveShardReplica 06:45:04,599 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveShardReplica 06:45:04,605 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@6c87870} 06:45:04,605 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} started 06:45:04,605 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:04,608 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:04,609 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:04,609 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveShardReplica 06:45:04,609 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveShardReplica 06:45:04,610 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica: Shard created, persistent : true 06:45:04,610 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#945223551] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:04,613 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica: Shard created, persistent : true 06:45:04,614 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 06:45:04,616 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#-1375152316 created and ready for shard:member-2-shard-cars-config_testRemoveShardReplica 06:45:04,616 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#-1375152316] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:04,616 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#-1554106218 created and ready for shard:member-2-shard-people-config_testRemoveShardReplica 06:45:04,618 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-cars-config_testRemoveShardReplica: Starting recovery with journal batch size 1 06:45:04,619 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveShardReplica: journal open: applyTo=0 06:45:04,621 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 06:45:04,621 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 06:45:04,622 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:04,622 PM [cluster-test-shard-dispatcher-14] [INFO] RaftActor - member-2-shard-people-config_testRemoveShardReplica: Starting recovery with journal batch size 1 06:45:04,622 PM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-cars-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:04,623 PM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveShardReplica: journal open: applyTo=0 06:45:04,625 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica#-1241307271] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:04,625 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#-1554106218] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:04,625 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#-839648975] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:04,626 PM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-cars-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 06:45:04,626 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:04,626 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:04,627 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 06:45:04,627 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveShardReplica is initialized 06:45:04,627 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 06:45:04,627 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 06:45:04,627 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:04,627 PM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:04,630 PM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-people-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 06:45:04,630 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:04,630 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:04,630 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , received role change from null to Follower 06:45:04,630 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 06:45:04,630 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveShardReplica is initialized 06:45:04,630 PM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 06:45:04,630 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 06:45:04,631 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveShardReplica 06:45:04,631 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveShardReplica from null to Follower 06:45:04,631 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardReplica from null to Follower 06:45:04,631 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveShardReplica 06:45:04,631 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@a3eef71} 06:45:04,632 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=1} started 06:45:04,632 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} 06:45:04,632 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:04,632 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveShardReplica 06:45:04,633 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveShardReplica 06:45:04,633 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Shard created, persistent : true 06:45:04,634 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-cars-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 06:45:04,634 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica-notifier#-2079226113 created and ready for shard:member-2-shard-cars-oper_testRemoveShardReplica 06:45:04,634 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica#-965902309] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:04,635 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica-notifier#-2079226113] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:04,635 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveShardReplica: journal open: applyTo=0 06:45:04,636 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica: Shard created, persistent : true 06:45:04,636 PM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica#1232490512] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:04,637 PM [cluster-test-shard-dispatcher-28] [INFO] RaftActor - member-2-shard-people-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 06:45:04,637 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica-notifier#-1125035172] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:04,637 PM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica-notifier#-1125035172 created and ready for shard:member-2-shard-people-oper_testRemoveShardReplica 06:45:04,638 PM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveShardReplica: journal open: applyTo=0 06:45:04,639 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:04,639 PM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-2-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:04,640 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:04,640 PM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-2-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:04,643 PM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-2-shard-cars-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 06:45:04,643 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 06:45:04,643 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveShardReplica is initialized 06:45:04,643 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:04,643 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:04,643 PM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 06:45:04,643 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 06:45:04,643 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveShardReplica from null to Follower 06:45:04,644 PM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-2-shard-people-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 06:45:04,644 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 06:45:04,644 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveShardReplica is initialized 06:45:04,644 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:04,644 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:04,644 PM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 06:45:04,644 PM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 06:45:04,644 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveShardReplica from null to Follower 06:45:04,673 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveShardReplica status sync done true 06:45:04,675 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 313, lastApplied : -1, commitIndex : -1 06:45:04,692 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveShardReplica status sync done true 06:45:04,693 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : -1, commitIndex : -1 06:45:04,803 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveShardReplica status sync done true 06:45:04,804 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 06:45:04,994 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 06:45:05,15 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 06:45:05,124 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 06:45:05,292 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 06:45:05,314 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 06:45:05,333 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 06:45:05,444 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 06:45:05,533 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2559] ignored due to non-matching UID, quarantine requested for [-882160981580280395] but current is [-303998725495193994]. Reason: ActorSystem terminated 06:45:05,534 PM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-882160981580280395] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 06:45:05,609 PM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-2-shard-cars-oper_testRemoveShardReplica (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-1-shard-cars-oper_testRemoveShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 0 - updating term 06:45:05,610 PM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-2-shard-people-oper_testRemoveShardReplica (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-1-shard-people-oper_testRemoveShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 0 - updating term 06:45:05,610 PM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-2-shard-cars-config_testRemoveShardReplica (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-1-shard-cars-config_testRemoveShardReplica, prevLogIndex=0, prevLogTerm=3, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 0 - updating term 06:45:05,609 PM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-2-shard-people-config_testRemoveShardReplica (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-1-shard-people-config_testRemoveShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 0 - updating term 06:45:05,612 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 06:45:05,613 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 06:45:05,615 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@34ca4424 06:45:05,615 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardReplica status sync done false 06:45:05,615 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardReplica status sync done true 06:45:05,615 PM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-2-shard-cars-config_testRemoveShardReplica (Follower): The followers log is empty and the senders prevLogIndex is 0 06:45:05,615 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardReplica status sync done false 06:45:05,615 PM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-2-shard-cars-config_testRemoveShardReplica (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply{term=3, success=false, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 06:45:05,615 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardReplica status sync done true 06:45:05,616 PM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-2-shard-cars-config_testRemoveShardReplica (Follower): The followers log is empty and the senders prevLogIndex is 0 06:45:05,616 PM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-2-shard-cars-config_testRemoveShardReplica (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply{term=3, success=false, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 06:45:05,616 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@499a2ef 06:45:05,616 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6d8edcec 06:45:05,616 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done false 06:45:05,616 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done true 06:45:05,618 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 1256, lastApplied : -1, commitIndex : -1 06:45:05,618 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 1133, lastApplied : -1, commitIndex : -1 06:45:05,618 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=false, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 1171, lastApplied : 0, commitIndex : 0 06:45:05,618 PM [cluster-test-shard-dispatcher-12] [INFO] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply{term=3, success=false, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, leader snapshotIndex: -1, snapshotTerm: -1, replicatedToAllIndex: -1 06:45:05,618 PM [cluster-test-shard-dispatcher-12] [INFO] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader): follower member-2-shard-cars-config_testRemoveShardReplica appears to be behind the leader from the last snapshot - updated: matchIndex: -1, nextIndex: 0 06:45:05,619 PM [cluster-test-shard-dispatcher-12] [INFO] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply{term=3, success=false, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, leader snapshotIndex: -1, snapshotTerm: -1, replicatedToAllIndex: -1 06:45:05,619 PM [cluster-test-shard-dispatcher-12] [INFO] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader): follower member-2-shard-cars-config_testRemoveShardReplica appears to be behind the leader from the last snapshot - updated: matchIndex: -1, nextIndex: 0 06:45:05,620 PM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveShardReplica: Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 06:45:05,620 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@63492f2d 06:45:05,620 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardReplica is ready 06:45:05,621 PM [cluster-test-shard-dispatcher-19] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 1244, lastApplied : -1, commitIndex : -1 06:45:05,621 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardReplica status sync done false 06:45:05,621 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardReplica status sync done true 06:45:05,621 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} is now ready 06:45:05,621 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#1749668930] 06:45:05,621 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 06:45:05,621 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#945223551] 06:45:05,622 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#945223551] 06:45:05,625 PM [main] [INFO] ClusterAdminRpcService - Removing replica for shard ShardName{value=cars} memberName MemberName{value=member-2}, datastoreType Config 06:45:05,625 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-2}] 06:45:05,625 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 06:45:05,626 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: LocalPrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-1858484956, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@5feacec7] 06:45:05,626 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-1858484956 for shard member-2-shard-cars-config_testRemoveShardReplica 06:45:05,627 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveShardReplica} for shard cars 06:45:05,627 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Leader shard successfully removed the replica shard cars 06:45:05,627 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ClusterAdminRpcService - Successfully removed replica for shard ShardName{value=cars} 06:45:05,627 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1001569632] 06:45:05,627 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:45:05,627 PM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-1858484956] 06:45:05,627 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-1858484956] 06:45:05,628 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#1749668930] 06:45:05,628 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 06:45:05,628 PM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#945223551] 06:45:05,628 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testRemoveShardReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]} has been successfully replicated to a majority of followers 06:45:05,629 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#945223551] with 27000 ms timeout 06:45:05,630 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2 : Local Shard replica for shard cars has been removed 06:45:05,630 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [people] 06:45:05,630 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica: Stopping Shard 06:45:05,631 PM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveShardReplica: journal closed 06:45:05,631 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#945223551] 06:45:05,632 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 06:45:05,653 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 06:45:05,678 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#1749668930] 06:45:05,678 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 06:45:05,678 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Local shard cars not found - shards present: [people] 06:45:05,679 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} 06:45:05,679 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveShardReplica 06:45:05,679 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 1 shards 06:45:05,680 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica: Stopping Shard 06:45:05,681 PM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveShardReplica: journal closed 06:45:05,681 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 06:45:05,681 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 06:45:05,682 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=1} 06:45:05,682 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveShardReplica 06:45:05,682 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveShardReplica 06:45:05,682 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 06:45:05,683 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Stopping Shard 06:45:05,683 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica: Stopping Shard 06:45:05,683 PM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveShardReplica: journal closed 06:45:05,683 PM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveShardReplica: journal closed 06:45:05,684 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 06:45:05,684 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 06:45:05,686 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:45:05,686 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:45:05,697 PM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:05,697 PM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:05,697 PM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:05,697 PM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:05,700 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 06:45:05,712 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} 06:45:05,713 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testRemoveShardReplica 06:45:05,713 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 06:45:05,713 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-people-config_testRemoveShardReplica: Stopping Shard 06:45:05,715 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-3-shard-people-config_testRemoveShardReplica: journal closed 06:45:05,715 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 06:45:05,715 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 06:45:05,715 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardReplica, generation=0} 06:45:05,716 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testRemoveShardReplica 06:45:05,716 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testRemoveShardReplica 06:45:05,716 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 06:45:05,716 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardReplica: Stopping Shard 06:45:05,716 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-people-oper_testRemoveShardReplica: Stopping Shard 06:45:05,717 PM [cluster-test-shard-dispatcher-34] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testRemoveShardReplica: journal closed 06:45:05,717 PM [cluster-test-shard-dispatcher-33] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testRemoveShardReplica: journal closed 06:45:05,717 PM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 06:45:05,717 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 06:45:05,718 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:45:05,719 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:45:06,700 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-303998725495193994] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 06:45:06,737 PM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:06,738 PM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:06,740 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 06:45:06,741 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [6211997111722349875] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated [WARN] [08/07/2025 18:45:06.742] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:45:06,755 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} 06:45:06,757 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testRemoveShardReplica 06:45:06,757 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testRemoveShardReplica 06:45:06,757 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 06:45:06,757 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@18bfd20c 06:45:06,757 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-config_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@18bfd20c 06:45:06,758 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@23331dd2 06:45:06,758 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@718b4bba 06:45:06,758 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica: Stopping Shard 06:45:06,758 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-people-config_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@718b4bba 06:45:06,758 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveShardReplica: pauseLeader successfully completed - doing transfer 06:45:06,759 PM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testRemoveShardReplica: journal closed 06:45:07,250 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveShardReplica: leader transfer aborted 06:45:07,250 PM [cluster-test-shard-dispatcher-19] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveShardReplica: Failed to transfer leadership in 492.8 ms 06:45:07,251 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: In unpauseLeader 06:45:07,251 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica: Stopping Shard 06:45:07,252 PM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-1-shard-people-config_testRemoveShardReplica: journal closed 06:45:07,253 PM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 06:45:07,253 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 06:45:07,253 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardReplica, generation=0} 06:45:07,254 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testRemoveShardReplica 06:45:07,254 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testRemoveShardReplica 06:45:07,254 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 06:45:07,255 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5faa1ee9 06:45:07,255 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@78d7a7ac 06:45:07,255 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@62d2012f 06:45:07,255 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@126964f5 06:45:07,255 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@62d2012f 06:45:07,255 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-oper_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@126964f5 06:45:07,255 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveShardReplica: pauseLeader successfully completed - doing transfer 06:45:07,255 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveShardReplica: pauseLeader successfully completed - doing transfer 06:45:07,681 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveShardReplica: leader transfer aborted 06:45:07,681 PM [cluster-test-shard-dispatcher-12] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveShardReplica: Failed to transfer leadership in 426.2 ms 06:45:07,681 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: In unpauseLeader 06:45:07,681 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Stopping Shard 06:45:07,682 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testRemoveShardReplica: journal closed 06:45:07,871 PM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 06:45:07,871 PM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] ShardManager - shard-manager-oper-member-1: An error occurred attempting to shut down the shards org.apache.pekko.pattern.AskTimeoutException: Ask timed out on [Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica#-1539618950]] after [600 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 06:45:07,872 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica: Stopping Shard 06:45:07,872 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveShardReplica: leader transfer aborted 06:45:07,872 PM [cluster-test-shard-dispatcher-19] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveShardReplica: Failed to transfer leadership in 617.5 ms 06:45:07,872 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: In unpauseLeader 06:45:07,873 PM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testRemoveShardReplica: journal closed 06:45:07,873 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 06:45:07,875 PM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:45:07,875 PM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:45:07,879 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/07/2025 18:45:07.881] [cluster-test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:45:07,908 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 06:45:07,921 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [1608228315151857152] 06:45:07,921 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 06:45:07,923 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 06:45:07,923 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 06:45:07,936 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:45:07,936 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:45:07,953 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 06:45:07,954 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 06:45:07,954 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 06:45:07,956 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:07,957 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:07,959 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveAllShardReplicas 06:45:07,963 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 06:45:07,964 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 06:45:07,964 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1813583923] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:07,966 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 06:45:07,966 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testRemoveAllShardReplicas 06:45:07,967 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testRemoveAllShardReplicas 06:45:07,967 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:07,967 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 06:45:07,967 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 06:45:07,970 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 06:45:07,970 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@63a8c668} 06:45:07,971 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} started 06:45:07,971 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:07,971 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testRemoveAllShardReplicas 06:45:07,971 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:07,972 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 06:45:07,972 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 06:45:07,972 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas on shard member-1-shard-cars-config_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1421255699] 06:45:07,972 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testRemoveAllShardReplicas 06:45:07,972 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 06:45:07,972 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 06:45:07,972 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas on shard member-1-shard-people-config_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#1625015317] 06:45:07,973 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:07,973 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Shard created, persistent : true 06:45:07,973 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1421255699] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:07,973 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas-notifier#-2016812885 created and ready for shard:member-1-shard-cars-config_testRemoveAllShardReplicas 06:45:07,973 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas-notifier#-2016812885] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:07,975 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Shard created, persistent : true 06:45:07,976 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas-notifier#-1617614153 created and ready for shard:member-1-shard-people-config_testRemoveAllShardReplicas 06:45:07,976 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-1-shard-people-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 06:45:07,978 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveAllShardReplicas 06:45:07,978 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 06:45:07,980 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testRemoveAllShardReplicas: journal open: applyTo=0 06:45:07,980 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-people-config_testRemoveAllShardReplicas: journal open: applyTo=0 06:45:07,982 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:07,982 PM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-cars-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:07,981 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#1625015317] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:07,983 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas-notifier#-1617614153] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:07,983 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#-1454907877] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:07,983 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:07,984 PM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-people-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:07,985 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 06:45:07,985 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 06:45:07,987 PM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-cars-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 06:45:07,987 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 06:45:07,987 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testRemoveAllShardReplicas is initialized 06:45:07,987 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 06:45:07,987 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:07,987 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , received role change from null to Follower 06:45:07,988 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 06:45:07,988 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 06:45:07,988 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveAllShardReplicas from null to Follower 06:45:07,987 PM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-people-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 06:45:07,987 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 06:45:07,989 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testRemoveAllShardReplicas 06:45:07,989 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} started 06:45:07,989 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testRemoveAllShardReplicas 06:45:07,989 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 06:45:07,989 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:07,989 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:07,989 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 06:45:07,989 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 06:45:07,989 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 06:45:07,989 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testRemoveAllShardReplicas is initialized 06:45:07,989 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@6ae48889} 06:45:07,989 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:07,989 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testRemoveAllShardReplicas 06:45:07,990 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 06:45:07,990 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 06:45:07,990 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas on shard member-1-shard-cars-oper_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas#-280031523] 06:45:07,990 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testRemoveAllShardReplicas 06:45:07,990 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Shard created, persistent : true 06:45:07,992 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActor - member-1-shard-cars-oper_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 06:45:07,992 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Shard created, persistent : true 06:45:07,992 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas-notifier#-2098640223 created and ready for shard:member-1-shard-people-oper_testRemoveAllShardReplicas 06:45:07,992 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , received role change from null to Follower 06:45:07,993 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-1-shard-people-oper_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 06:45:07,993 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 06:45:07,993 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas-notifier#-1094193525 created and ready for shard:member-1-shard-cars-oper_testRemoveAllShardReplicas 06:45:07,993 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas#-280031523] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:07,993 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas#1921830553] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:07,993 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas-notifier#-2098640223] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:07,993 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas-notifier#-1094193525] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:07,994 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testRemoveAllShardReplicas: journal open: applyTo=0 06:45:07,994 PM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testRemoveAllShardReplicas: journal open: applyTo=0 06:45:07,994 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:07,995 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:07,995 PM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-people-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:07,995 PM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-1-shard-cars-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:07,996 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 06:45:07,996 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 06:45:07,996 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas on shard member-1-shard-people-oper_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas#1921830553] 06:45:07,996 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveAllShardReplicas from null to Follower 06:45:07,998 PM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-people-oper_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 06:45:07,998 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 06:45:07,998 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testRemoveAllShardReplicas is initialized 06:45:07,998 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 06:45:07,998 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:07,998 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , received role change from null to Follower 06:45:07,998 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 06:45:08,1 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveAllShardReplicas from null to Follower 06:45:08,1 PM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-1-shard-cars-oper_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 06:45:08,1 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 06:45:08,1 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:08,1 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , received role change from null to Follower 06:45:08,2 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 06:45:08,2 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testRemoveAllShardReplicas is initialized 06:45:08,2 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 06:45:08,2 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveAllShardReplicas from null to Follower 06:45:08,9 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 06:45:08,22 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-7445288863573689474] 06:45:08,22 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 06:45:08,25 PM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:45:08,25 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 06:45:08,26 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:45:08,26 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:45:08,39 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:08,40 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:08,42 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveAllShardReplicas 06:45:08,57 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1612636903] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:08,58 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 06:45:08,58 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 06:45:08,61 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 06:45:08,61 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveAllShardReplicas 06:45:08,61 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveAllShardReplicas 06:45:08,63 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 06:45:08,64 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@aed2525} 06:45:08,64 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} started 06:45:08,65 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:08,67 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:08,68 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:08,68 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveAllShardReplicas 06:45:08,68 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveAllShardReplicas 06:45:08,69 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Shard created, persistent : true 06:45:08,70 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-cars-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 06:45:08,70 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveAllShardReplicas 06:45:08,71 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 06:45:08,71 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 06:45:08,72 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Shard created, persistent : true 06:45:08,73 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveAllShardReplicas: journal open: applyTo=0 06:45:08,76 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas-notifier#2060918353 created and ready for shard:member-2-shard-cars-config_testRemoveAllShardReplicas 06:45:08,76 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#-784575948] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:08,77 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#-2039692704] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:08,77 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas#-1690449167] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:08,77 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas-notifier#2060918353] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:08,78 PM [cluster-test-shard-dispatcher-14] [INFO] RaftActor - member-2-shard-people-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 06:45:08,79 PM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveAllShardReplicas: journal open: applyTo=0 06:45:08,80 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:08,80 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 06:45:08,80 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveAllShardReplicas 06:45:08,80 PM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-2-shard-cars-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:08,81 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveAllShardReplicas 06:45:08,82 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:08,82 PM [cluster-test-shard-dispatcher-22] [INFO] PekkoRecovery - member-2-shard-people-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:08,85 PM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-2-shard-cars-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 06:45:08,85 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 06:45:08,85 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveAllShardReplicas is initialized 06:45:08,85 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 06:45:08,85 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:08,85 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveAllShardReplicas , received role change from null to Follower 06:45:08,85 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 06:45:08,85 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveAllShardReplicas from null to Follower 06:45:08,85 PM [cluster-test-shard-dispatcher-22] [INFO] PekkoRecovery - member-2-shard-people-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 06:45:08,86 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 06:45:08,86 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:08,86 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 06:45:08,86 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveAllShardReplicas is initialized 06:45:08,88 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas-notifier#2010030655 created and ready for shard:member-2-shard-people-config_testRemoveAllShardReplicas 06:45:08,88 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveAllShardReplicas , received role change from null to Follower 06:45:08,88 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 06:45:08,88 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas-notifier#2010030655] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:08,88 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveAllShardReplicas from null to Follower 06:45:08,97 PM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 06:45:08,98 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@560f4aef} 06:45:08,98 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} started 06:45:08,100 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:08,100 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveAllShardReplicas 06:45:08,101 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveAllShardReplicas 06:45:08,101 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Shard created, persistent : true 06:45:08,101 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Shard created, persistent : true 06:45:08,102 PM [cluster-test-shard-dispatcher-14] [INFO] RaftActor - member-2-shard-cars-oper_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 06:45:08,102 PM [cluster-test-shard-dispatcher-22] [INFO] RaftActor - member-2-shard-people-oper_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 06:45:08,102 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas#1111118547] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:08,102 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas#1423350570] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:08,102 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas-notifier#-1065110012 created and ready for shard:member-2-shard-cars-oper_testRemoveAllShardReplicas 06:45:08,103 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas-notifier#-1065110012] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:08,103 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas-notifier#1771599560 created and ready for shard:member-2-shard-people-oper_testRemoveAllShardReplicas 06:45:08,103 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas-notifier#1771599560] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:08,103 PM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveAllShardReplicas: journal open: applyTo=0 06:45:08,103 PM [cluster-test-shard-dispatcher-22] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveAllShardReplicas: journal open: applyTo=0 06:45:08,106 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:08,106 PM [cluster-test-shard-dispatcher-22] [INFO] PekkoRecovery - member-2-shard-cars-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:08,106 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:08,106 PM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-people-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:08,109 PM [cluster-test-shard-dispatcher-22] [INFO] PekkoRecovery - member-2-shard-cars-oper_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 06:45:08,109 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 06:45:08,109 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveAllShardReplicas is initialized 06:45:08,109 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 06:45:08,109 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:08,109 PM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveAllShardReplicas , received role change from null to Follower 06:45:08,109 PM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 06:45:08,109 PM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-people-oper_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 06:45:08,110 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 06:45:08,110 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:08,110 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveAllShardReplicas , received role change from null to Follower 06:45:08,110 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 06:45:08,110 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveAllShardReplicas is initialized 06:45:08,110 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 06:45:08,110 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveAllShardReplicas from null to Follower 06:45:08,110 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveAllShardReplicas from null to Follower 06:45:08,120 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 06:45:08,125 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 06:45:08,127 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 06:45:08,127 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:08,127 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 06:45:08,127 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 06:45:08,127 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 06:45:08,127 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 06:45:08,128 PM [cluster-test-shard-dispatcher-14] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveAllShardReplicas: Peer address for peer member-1-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 06:45:08,128 PM [cluster-test-shard-dispatcher-14] [INFO] PeerInfos - member-2-shard-people-config_testRemoveAllShardReplicas: Peer address for peer member-1-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 06:45:08,128 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:08,128 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 06:45:08,128 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 06:45:08,128 PM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveAllShardReplicas: Peer address for peer member-1-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 06:45:08,134 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 06:45:08,134 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 06:45:08,134 PM [cluster-test-shard-dispatcher-32] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveAllShardReplicas: Peer address for peer member-1-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 06:45:08,142 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-2047525980282485577] 06:45:08,143 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 06:45:08,145 PM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:45:08,145 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 06:45:08,146 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:45:08,146 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:45:08,160 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:08,166 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:08,169 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveAllShardReplicas 06:45:08,181 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 06:45:08,181 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 06:45:08,181 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#191728095] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:08,184 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 06:45:08,184 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testRemoveAllShardReplicas 06:45:08,184 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testRemoveAllShardReplicas 06:45:08,187 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 06:45:08,188 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} started 06:45:08,188 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:08,188 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@4a22f802} 06:45:08,189 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:08,189 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testRemoveAllShardReplicas 06:45:08,190 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: Shard created, persistent : true 06:45:08,190 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#1958433673] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:08,190 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:08,190 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas-notifier#1188625585 created and ready for shard:member-3-shard-cars-config_testRemoveAllShardReplicas 06:45:08,191 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testRemoveAllShardReplicas 06:45:08,193 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveAllShardReplicas 06:45:08,193 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas-notifier#1188625585] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:08,195 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-3-shard-cars-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 06:45:08,196 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testRemoveAllShardReplicas: journal open: applyTo=0 06:45:08,197 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: Shard created, persistent : true 06:45:08,197 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-3-shard-people-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 06:45:08,198 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-3-shard-people-config_testRemoveAllShardReplicas: journal open: applyTo=0 06:45:08,199 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas-notifier#401522625 created and ready for shard:member-3-shard-people-config_testRemoveAllShardReplicas 06:45:08,199 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 06:45:08,199 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 06:45:08,199 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas#-1773245923] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:08,199 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#-1607211330] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:08,199 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas-notifier#401522625] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:08,203 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:08,204 PM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-3-shard-cars-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:08,207 PM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-3-shard-cars-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 06:45:08,208 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 06:45:08,208 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:08,208 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 06:45:08,208 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testRemoveAllShardReplicas is initialized 06:45:08,212 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 06:45:08,212 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testRemoveAllShardReplicas 06:45:08,212 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testRemoveAllShardReplicas 06:45:08,214 PM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveAllShardReplicas , received role change from null to Follower 06:45:08,214 PM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 06:45:08,214 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveAllShardReplicas from null to Follower 06:45:08,216 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 06:45:08,217 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:08,217 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@44e51f36} 06:45:08,217 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 06:45:08,219 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} started 06:45:08,219 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 06:45:08,219 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:08,219 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testRemoveAllShardReplicas 06:45:08,220 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:08,220 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 06:45:08,220 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 06:45:08,220 PM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-3-shard-cars-config_testRemoveAllShardReplicas: Peer address for peer member-1-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 06:45:08,221 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: Shard created, persistent : true 06:45:08,221 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas-notifier#29534053 created and ready for shard:member-3-shard-cars-oper_testRemoveAllShardReplicas 06:45:08,221 PM [cluster-test-shard-dispatcher-26] [INFO] RaftActor - member-3-shard-cars-oper_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 06:45:08,221 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 06:45:08,221 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#1383682839] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:08,221 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas-notifier#29534053] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:08,222 PM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-3-shard-people-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:08,222 PM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testRemoveAllShardReplicas: journal open: applyTo=0 06:45:08,223 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 06:45:08,223 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 06:45:08,224 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:08,224 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testRemoveAllShardReplicas 06:45:08,224 PM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-cars-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:08,225 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:08,225 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 06:45:08,225 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 06:45:08,225 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 06:45:08,225 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 06:45:08,225 PM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#108616346] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:08,225 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: Shard created, persistent : true 06:45:08,226 PM [cluster-test-shard-dispatcher-26] [INFO] RaftActor - member-3-shard-people-oper_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 06:45:08,226 PM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas-notifier#-646725852 created and ready for shard:member-3-shard-people-oper_testRemoveAllShardReplicas 06:45:08,226 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas-notifier#-646725852] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:08,226 PM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-3-shard-people-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 06:45:08,227 PM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testRemoveAllShardReplicas: journal open: applyTo=0 06:45:08,227 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 06:45:08,227 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testRemoveAllShardReplicas is initialized 06:45:08,227 PM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-cars-oper_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 06:45:08,227 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 06:45:08,227 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:08,227 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveAllShardReplicas , received role change from null to Follower 06:45:08,227 PM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-3-shard-people-config_testRemoveAllShardReplicas: Peer address for peer member-1-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 06:45:08,227 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 06:45:08,227 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testRemoveAllShardReplicas from null to Follower 06:45:08,228 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 06:45:08,228 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testRemoveAllShardReplicas is initialized 06:45:08,228 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:08,228 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 06:45:08,228 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:08,228 PM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-3-shard-people-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:08,228 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveAllShardReplicas , received role change from null to Follower 06:45:08,228 PM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-cars-oper_testRemoveAllShardReplicas: Peer address for peer member-1-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 06:45:08,228 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 06:45:08,228 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testRemoveAllShardReplicas from null to Follower 06:45:08,230 PM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-3-shard-people-oper_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 06:45:08,230 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 06:45:08,230 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:08,231 PM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-3-shard-people-oper_testRemoveAllShardReplicas: Peer address for peer member-1-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 06:45:08,231 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 06:45:08,231 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testRemoveAllShardReplicas is initialized 06:45:08,231 PM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveAllShardReplicas , received role change from null to Follower 06:45:08,231 PM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 06:45:08,231 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testRemoveAllShardReplicas from null to Follower 06:45:08,320 PM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Starting new election term 1 06:45:08,320 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:45:08,320 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 06:45:08,320 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:08,320 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , received role change from Follower to Candidate 06:45:08,320 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveAllShardReplicas from Follower to Candidate 06:45:08,349 PM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Starting new election term 1 06:45:08,349 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:45:08,349 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 06:45:08,349 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:08,349 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , received role change from Follower to Candidate 06:45:08,349 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveAllShardReplicas from Follower to Candidate 06:45:08,379 PM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 1 06:45:08,379 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:45:08,379 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 06:45:08,379 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:08,379 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , received role change from Follower to Candidate 06:45:08,379 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveAllShardReplicas from Follower to Candidate 06:45:08,409 PM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 1 06:45:08,410 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:45:08,410 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 06:45:08,410 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:08,410 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , received role change from Follower to Candidate 06:45:08,410 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveAllShardReplicas from Follower to Candidate 06:45:08,700 PM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Starting new election term 2 06:45:08,730 PM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 2 06:45:08,750 PM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Starting new election term 2 06:45:08,819 PM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 2 06:45:08,938 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:08,938 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 06:45:08,938 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 06:45:08,938 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 06:45:08,938 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 06:45:08,938 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:08,938 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 06:45:08,938 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 06:45:08,939 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 06:45:08,939 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 06:45:08,939 PM [cluster-test-shard-dispatcher-22] [INFO] PeerInfos - member-1-shard-cars-oper_testRemoveAllShardReplicas: Peer address for peer member-3-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 06:45:08,939 PM [cluster-test-shard-dispatcher-22] [INFO] PeerInfos - member-1-shard-cars-oper_testRemoveAllShardReplicas: Peer address for peer member-2-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 06:45:08,939 PM [cluster-test-shard-dispatcher-22] [INFO] PeerInfos - member-1-shard-people-oper_testRemoveAllShardReplicas: Peer address for peer member-3-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 06:45:08,939 PM [cluster-test-shard-dispatcher-22] [INFO] PeerInfos - member-1-shard-people-oper_testRemoveAllShardReplicas: Peer address for peer member-2-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 06:45:08,939 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:08,939 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 06:45:08,939 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 06:45:08,939 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 06:45:08,939 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 06:45:08,939 PM [cluster-test-shard-dispatcher-22] [INFO] PeerInfos - member-1-shard-cars-config_testRemoveAllShardReplicas: Peer address for peer member-3-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 06:45:08,939 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 06:45:08,939 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 06:45:08,939 PM [cluster-test-shard-dispatcher-22] [INFO] PeerInfos - member-1-shard-people-config_testRemoveAllShardReplicas: Peer address for peer member-3-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 06:45:08,939 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:08,939 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 06:45:08,939 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 06:45:08,939 PM [cluster-test-shard-dispatcher-22] [INFO] PeerInfos - member-1-shard-cars-config_testRemoveAllShardReplicas: Peer address for peer member-2-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 06:45:08,939 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 06:45:08,939 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 06:45:08,939 PM [cluster-test-shard-dispatcher-22] [INFO] PeerInfos - member-1-shard-people-config_testRemoveAllShardReplicas: Peer address for peer member-2-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 06:45:08,945 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:08,945 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:08,945 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 06:45:08,945 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 06:45:08,945 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 06:45:08,945 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 06:45:08,945 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 06:45:08,945 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 06:45:08,945 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 06:45:08,945 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 06:45:08,945 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:08,945 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 06:45:08,945 PM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveAllShardReplicas: Peer address for peer member-3-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 06:45:08,945 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 06:45:08,945 PM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveAllShardReplicas: Peer address for peer member-3-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 06:45:08,945 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 06:45:08,945 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 06:45:08,945 PM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-2-shard-people-config_testRemoveAllShardReplicas: Peer address for peer member-3-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 06:45:08,948 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:08,948 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 06:45:08,948 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 06:45:08,948 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 06:45:08,948 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 06:45:08,948 PM [cluster-test-shard-dispatcher-22] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveAllShardReplicas: Peer address for peer member-3-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 06:45:09,59 PM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Starting new election term 3 06:45:09,61 PM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:09,63 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:09,63 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:09,63 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 06:45:09,63 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 06:45:09,63 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 06:45:09,63 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 06:45:09,63 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 06:45:09,63 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 06:45:09,63 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 06:45:09,63 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 06:45:09,63 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:09,63 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:09,63 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 06:45:09,63 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 06:45:09,63 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 06:45:09,63 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 06:45:09,63 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 06:45:09,63 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 06:45:09,63 PM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-3-shard-cars-oper_testRemoveAllShardReplicas: Peer address for peer member-2-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 06:45:09,63 PM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-cars-config_testRemoveAllShardReplicas: Peer address for peer member-2-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 06:45:09,63 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 06:45:09,63 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 06:45:09,63 PM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-3-shard-people-oper_testRemoveAllShardReplicas: Peer address for peer member-2-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 06:45:09,64 PM [cluster-test-shard-dispatcher-22] [INFO] Follower - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:09,69 PM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-3-shard-people-config_testRemoveAllShardReplicas: Peer address for peer member-2-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 06:45:09,71 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 06:45:09,71 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 06:45:09,71 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: became leader with frontend state for [] 06:45:09,71 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , received role change from Candidate to Leader 06:45:09,71 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@67f317b0 06:45:09,71 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveAllShardReplicas from Candidate to Leader 06:45:09,72 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveAllShardReplicas status sync done false 06:45:09,72 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1d5749e8 06:45:09,73 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveAllShardReplicas status sync done false 06:45:09,73 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5ddb670d 06:45:09,91 PM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Starting new election term 3 06:45:09,92 PM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:09,93 PM [cluster-test-shard-dispatcher-34] [INFO] Follower - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:09,102 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 06:45:09,102 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 06:45:09,102 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: became leader with frontend state for [] 06:45:09,102 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , received role change from Candidate to Leader 06:45:09,102 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@63d06a9d 06:45:09,102 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveAllShardReplicas from Candidate to Leader 06:45:09,102 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 06:45:09,102 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} is now ready 06:45:09,102 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1813583923] 06:45:09,103 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:45:09,103 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1421255699] 06:45:09,103 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1421255699] 06:45:09,103 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1612636903] 06:45:09,103 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 06:45:09,104 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#-784575948] 06:45:09,104 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#-784575948] 06:45:09,104 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@18d9f2d2 06:45:09,104 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 06:45:09,104 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@b594c5a 06:45:09,104 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 06:45:09,104 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#191728095] 06:45:09,105 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveAllShardReplicas status sync done false 06:45:09,105 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 06:45:09,105 PM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#1958433673] 06:45:09,105 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#1958433673] 06:45:09,105 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveAllShardReplicas status sync done false 06:45:09,109 PM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 3 06:45:09,110 PM [cluster-test-shard-dispatcher-22] [INFO] Follower - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:09,111 PM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:09,116 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@458db110] 06:45:09,117 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1 doCreateShard: shardId: member-1-shard-pets-config_testRemoveAllShardReplicas, memberNames: [MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], peerAddresses: {member-2-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas, member-3-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas}, isActiveMember: true 06:45:09,118 PM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 06:45:09,118 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Shard created, persistent : true 06:45:09,118 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 06:45:09,118 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: became leader with frontend state for [] 06:45:09,118 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , received role change from Candidate to Leader 06:45:09,118 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5ddf54ee 06:45:09,119 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveAllShardReplicas from Candidate to Leader 06:45:09,119 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas-notifier#1601384906 created and ready for shard:member-1-shard-pets-config_testRemoveAllShardReplicas 06:45:09,119 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActor - member-1-shard-pets-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 06:45:09,120 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveAllShardReplicas status sync done false 06:45:09,120 PM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-pets-config_testRemoveAllShardReplicas: journal open: applyTo=0 06:45:09,120 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@42acef47 06:45:09,121 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@57d64550 06:45:09,121 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:09,121 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveAllShardReplicas status sync done false 06:45:09,121 PM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-pets-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:09,124 PM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-pets-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 06:45:09,124 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 06:45:09,124 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:09,124 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , received role change from null to Follower 06:45:09,124 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [pets] 06:45:09,124 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-pets-config_testRemoveAllShardReplicas is initialized 06:45:09,124 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 06:45:09,124 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testRemoveAllShardReplicas from null to Follower 06:45:09,128 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@15b53732] 06:45:09,129 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2 doCreateShard: shardId: member-2-shard-pets-config_testRemoveAllShardReplicas, memberNames: [MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], peerAddresses: {member-1-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas, member-3-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas}, isActiveMember: true 06:45:09,130 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Shard created, persistent : true 06:45:09,131 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas-notifier#-945173196 created and ready for shard:member-2-shard-pets-config_testRemoveAllShardReplicas 06:45:09,131 PM [cluster-test-shard-dispatcher-34] [INFO] RaftActor - member-2-shard-pets-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 06:45:09,133 PM [cluster-test-shard-dispatcher-34] [INFO] EnabledRaftStorage - member-2-shard-pets-config_testRemoveAllShardReplicas: journal open: applyTo=0 06:45:09,134 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:09,134 PM [cluster-test-shard-dispatcher-34] [INFO] PekkoRecovery - member-2-shard-pets-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:09,139 PM [cluster-test-shard-dispatcher-34] [INFO] PekkoRecovery - member-2-shard-pets-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 06:45:09,139 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [pets] 06:45:09,139 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-pets-config_testRemoveAllShardReplicas is initialized 06:45:09,140 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 06:45:09,140 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:09,140 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testRemoveAllShardReplicas , received role change from null to Follower 06:45:09,140 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 06:45:09,140 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-pets-config_testRemoveAllShardReplicas from null to Follower 06:45:09,141 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@47007704] 06:45:09,142 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3 doCreateShard: shardId: member-3-shard-pets-config_testRemoveAllShardReplicas, memberNames: [MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], peerAddresses: {member-2-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas, member-1-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas}, isActiveMember: true 06:45:09,143 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: Shard created, persistent : true 06:45:09,144 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1813583923] 06:45:09,144 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : pets 06:45:09,144 PM [cluster-test-shard-dispatcher-29] [INFO] RaftActor - member-3-shard-pets-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 06:45:09,144 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#1373464546] 06:45:09,144 PM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas-notifier#-44348485 created and ready for shard:member-3-shard-pets-config_testRemoveAllShardReplicas 06:45:09,144 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#1373464546] 06:45:09,145 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1612636903] 06:45:09,146 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : pets 06:45:09,146 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-2142720513] 06:45:09,147 PM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-pets-config_testRemoveAllShardReplicas: journal open: applyTo=0 06:45:09,147 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-2142720513] 06:45:09,148 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#191728095] 06:45:09,148 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:09,148 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : pets 06:45:09,148 PM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-pets-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:09,151 PM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-pets-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 06:45:09,152 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 06:45:09,152 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:09,152 PM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-pets-config_testRemoveAllShardReplicas , received role change from null to Follower 06:45:09,149 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 300000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-pets-config_testRemoveAllShardReplicas, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 06:45:09,155 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [pets] 06:45:09,155 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-pets-config_testRemoveAllShardReplicas is initialized 06:45:09,155 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-pets-config_testRemoveAllShardReplicas is initialized - notifying 1 OnShardInitialized callbacks 06:45:09,155 PM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#-762075496] 06:45:09,155 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#-762075496] 06:45:09,155 PM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-pets-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 06:45:09,155 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-pets-config_testRemoveAllShardReplicas from null to Follower 06:45:09,157 PM [main] [INFO] ClusterAdminRpcService - Removing replicas for all shards 06:45:09,157 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people, pets] for data store config_testRemoveAllShardReplicas 06:45:09,157 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-3}] 06:45:09,157 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testRemoveAllShardReplicas 06:45:09,158 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-3}] 06:45:09,158 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=people, memberName=MemberName{name=member-3}] 06:45:09,158 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=pets, memberName=MemberName{name=member-3}] 06:45:09,158 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 06:45:09,158 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=people, memberName=MemberName{name=member-3}] 06:45:09,158 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas, primaryVersion=13] 06:45:09,158 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 06:45:09,158 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for people: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas, primaryVersion=13] 06:45:09,158 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=pets, waitUntilReady=true] 06:45:09,159 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 18000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-pets-config_testRemoveAllShardReplicas, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=null, activeMember=true] 06:45:09,159 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas for shard member-3-shard-cars-config_testRemoveAllShardReplicas 06:45:09,159 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas for shard member-3-shard-people-config_testRemoveAllShardReplicas 06:45:09,159 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 06:45:09,159 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas, primaryVersion=13] 06:45:09,160 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 06:45:09,160 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Scheduling 18000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-people-oper_testRemoveAllShardReplicas, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=null, activeMember=true] 06:45:09,160 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas for shard member-3-shard-cars-oper_testRemoveAllShardReplicas 06:45:09,163 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testRemoveAllShardReplicas} for shard cars 06:45:09,163 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully removed the replica shard cars 06:45:09,163 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveAllShardReplicas} for shard cars 06:45:09,163 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard cars 06:45:09,163 PM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 06:45:09,163 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-people-config_testRemoveAllShardReplicas} for shard people 06:45:09,163 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard people 06:45:09,163 PM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 1 06:45:09,163 PM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 06:45:09,163 PM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 2 06:45:09,163 PM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 06:45:09,163 PM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 3 06:45:09,166 PM [cluster-test-shard-dispatcher-19] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]} has been successfully replicated to a majority of followers 06:45:09,167 PM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 06:45:09,168 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#1958433673] with 27000 ms timeout 06:45:09,168 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard cars has been removed 06:45:09,168 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [pets, people] 06:45:09,169 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveAllShardReplicas status sync done true 06:45:09,169 PM [cluster-test-shard-dispatcher-19] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-config_testRemoveAllShardReplicas: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]} has been successfully replicated to a majority of followers 06:45:09,169 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveAllShardReplicas status sync done true 06:45:09,169 PM [cluster-test-shard-dispatcher-19] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]} has been successfully replicated to a majority of followers 06:45:09,171 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveAllShardReplicas status sync done true 06:45:09,171 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: Stopping Shard 06:45:09,172 PM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testRemoveAllShardReplicas: journal closed 06:45:09,172 PM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#1958433673] 06:45:09,172 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas#-1773245923] with 27000 ms timeout 06:45:09,172 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#1383682839] with 27000 ms timeout 06:45:09,173 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard people has been removed 06:45:09,173 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [pets] 06:45:09,173 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: Stopping Shard 06:45:09,173 PM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-3-shard-people-config_testRemoveAllShardReplicas: journal closed 06:45:09,174 PM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas#-1773245923] 06:45:09,174 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3 : Local Shard replica for shard cars has been removed 06:45:09,174 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [people] 06:45:09,176 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: Stopping Shard 06:45:09,176 PM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testRemoveAllShardReplicas: journal closed 06:45:09,176 PM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#1383682839] 06:45:09,189 PM [cluster-test-shard-dispatcher-22] [INFO] Candidate - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 3 06:45:09,191 PM [cluster-test-shard-dispatcher-33] [INFO] Follower - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:09,191 PM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:09,198 PM [cluster-test-shard-dispatcher-22] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 06:45:09,198 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 06:45:09,198 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: became leader with frontend state for [] 06:45:09,198 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , received role change from Candidate to Leader 06:45:09,198 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1b34ba50 06:45:09,199 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveAllShardReplicas from Candidate to Leader 06:45:09,199 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveAllShardReplicas is ready 06:45:09,200 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveAllShardReplicas status sync done false 06:45:09,200 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveAllShardReplicas status sync done false 06:45:09,200 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@22daebba 06:45:09,200 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveAllShardReplicas is ready 06:45:09,200 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@173f7e7c 06:45:09,200 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testRemoveAllShardReplicas is ready - notifying 1 OnShardInitialized callbacks 06:45:09,200 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Found primary for people: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas, primaryVersion=13] 06:45:09,200 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testRemoveAllShardReplicas is ready 06:45:09,201 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas for shard member-3-shard-people-oper_testRemoveAllShardReplicas 06:45:09,206 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-people-oper_testRemoveAllShardReplicas} for shard people 06:45:09,206 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully removed the replica shard people 06:45:09,206 PM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 06:45:09,206 PM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 4 06:45:09,210 PM [cluster-test-shard-dispatcher-22] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]} has been successfully replicated to a majority of followers 06:45:09,211 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveAllShardReplicas status sync done true 06:45:09,211 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#108616346] with 27000 ms timeout 06:45:09,211 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3 : Local Shard replica for shard people has been removed 06:45:09,211 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [] 06:45:09,211 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: Stopping Shard 06:45:09,212 PM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testRemoveAllShardReplicas: journal closed 06:45:09,212 PM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#108616346] 06:45:09,521 PM [cluster-test-shard-dispatcher-22] [INFO] Candidate - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): Starting new election term 1 06:45:09,521 PM [cluster-test-shard-dispatcher-22] [INFO] RaftActorBehavior - member-1-shard-pets-config_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:45:09,521 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 06:45:09,521 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:09,521 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , received role change from Follower to Candidate 06:45:09,521 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testRemoveAllShardReplicas from Follower to Candidate 06:45:09,522 PM [cluster-test-shard-dispatcher-22] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveAllShardReplicas, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 314, lastApplied : 0, commitIndex : 0 06:45:09,523 PM [cluster-test-shard-dispatcher-33] [INFO] Follower - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Term 1 in "RequestVote{term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:09,525 PM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): Term 1 in "RequestVote{term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:09,531 PM [cluster-test-shard-dispatcher-22] [INFO] RaftActorBehavior - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 06:45:09,531 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 06:45:09,531 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: became leader with frontend state for [] 06:45:09,531 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , received role change from Candidate to Leader 06:45:09,531 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@501f9852 06:45:09,531 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testRemoveAllShardReplicas from Candidate to Leader 06:45:09,531 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 06:45:09,532 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5fe82746 06:45:09,532 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-pets-config_testRemoveAllShardReplicas is ready - notifying 1 OnShardInitialized callbacks 06:45:09,532 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for pets: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas, primaryVersion=13] 06:45:09,532 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 06:45:09,532 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-pets-config_testRemoveAllShardReplicas status sync done false 06:45:09,532 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testRemoveAllShardReplicas status sync done false 06:45:09,532 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas for shard member-3-shard-pets-config_testRemoveAllShardReplicas 06:45:09,533 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2fb8f9ed 06:45:09,533 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 06:45:09,536 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-pets-config_testRemoveAllShardReplicas} for shard pets 06:45:09,536 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard pets 06:45:09,536 PM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=pets}, type Config 06:45:09,536 PM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 5 06:45:09,538 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1813583923] 06:45:09,538 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:45:09,538 PM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1421255699] 06:45:09,538 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1421255699] 06:45:09,538 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1813583923] 06:45:09,539 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 06:45:09,539 PM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#1625015317] 06:45:09,539 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#1625015317] 06:45:09,539 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1813583923] 06:45:09,539 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : pets 06:45:09,539 PM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#1373464546] 06:45:09,539 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#1373464546] 06:45:09,539 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1612636903] 06:45:09,539 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 06:45:09,540 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#-784575948] 06:45:09,540 PM [cluster-test-shard-dispatcher-22] [INFO] RaftActorVotingConfigSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]} has been successfully replicated to a majority of followers 06:45:09,540 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#-784575948] 06:45:09,540 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1612636903] 06:45:09,540 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 06:45:09,540 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas#-1690449167] 06:45:09,540 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testRemoveAllShardReplicas status sync done true 06:45:09,540 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas#-1690449167] 06:45:09,540 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1612636903] 06:45:09,540 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : pets 06:45:09,541 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-2142720513] 06:45:09,541 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#-762075496] with 27000 ms timeout 06:45:09,541 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard pets has been removed 06:45:09,541 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [] 06:45:09,541 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-2142720513] 06:45:09,541 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: Stopping Shard 06:45:09,541 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#191728095] 06:45:09,542 PM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-3-shard-pets-config_testRemoveAllShardReplicas: journal closed 06:45:09,542 PM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#-762075496] 06:45:09,543 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 06:45:09,543 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard cars not found - shards present: [] 06:45:09,543 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#191728095] 06:45:09,543 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 06:45:09,543 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard people not found - shards present: [] 06:45:09,544 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#191728095] 06:45:09,544 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : pets 06:45:09,544 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard pets not found - shards present: [] 06:45:09,544 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 06:45:09,544 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 0 shards 06:45:09,545 PM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 06:45:09,545 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 06:45:09,545 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 06:45:09,545 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 0 shards 06:45:09,545 PM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 06:45:09,546 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 06:45:09,547 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:45:09,547 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:45:09,552 PM [cluster-test-pekko.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:09,552 PM [cluster-test-pekko.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:09,552 PM [cluster-test-pekko.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:09,552 PM [cluster-test-pekko.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:09,555 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 06:45:09,572 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 06:45:09,573 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-pets-config_testRemoveAllShardReplicas 06:45:09,573 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testRemoveAllShardReplicas 06:45:09,573 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveAllShardReplicas 06:45:09,573 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 3 shards 06:45:09,573 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Stopping Shard 06:45:09,573 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Stopping Shard 06:45:09,574 PM [cluster-test-shard-dispatcher-22] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveAllShardReplicas: journal closed 06:45:09,574 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Stopping Shard 06:45:09,574 PM [cluster-test-shard-dispatcher-34] [INFO] EnabledRaftStorage - member-2-shard-pets-config_testRemoveAllShardReplicas: journal closed 06:45:09,574 PM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveAllShardReplicas: journal closed 06:45:09,575 PM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 06:45:09,575 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 06:45:09,575 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 06:45:09,575 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveAllShardReplicas 06:45:09,575 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveAllShardReplicas 06:45:09,576 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 06:45:09,576 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Stopping Shard 06:45:09,576 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Stopping Shard 06:45:09,576 PM [cluster-test-shard-dispatcher-34] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveAllShardReplicas: journal closed 06:45:09,577 PM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveAllShardReplicas: journal closed 06:45:09,577 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 06:45:09,577 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 06:45:09,578 PM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:45:09,578 PM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:45:09,957 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 06:45:10,567 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-2047525980282485577] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 06:45:10,597 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-7445288863573689474] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 06:45:10,599 PM [cluster-test-pekko.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:10,599 PM [cluster-test-pekko.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:10,602 PM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/07/2025 18:45:10.605] [cluster-test-pekko.actor.internal-dispatcher-7] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:45:10,616 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 06:45:10,617 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-pets-config_testRemoveAllShardReplicas 06:45:10,617 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testRemoveAllShardReplicas 06:45:10,617 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testRemoveAllShardReplicas 06:45:10,617 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 3 shards 06:45:10,617 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@54382854 06:45:10,617 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6271375d 06:45:10,617 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@78207eba 06:45:10,617 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5affcca3 06:45:10,617 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-pets-config_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6271375d 06:45:10,617 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-config_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@78207eba 06:45:10,617 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 06:45:10,617 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 06:45:10,617 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@68791e7f 06:45:10,618 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@298f6b27 06:45:10,618 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-config_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@298f6b27 06:45:10,618 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 06:45:10,986 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveAllShardReplicas: leader transfer aborted 06:45:10,986 PM [cluster-test-shard-dispatcher-19] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveAllShardReplicas: Failed to transfer leadership in 368.8 ms 06:45:10,986 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: In unpauseLeader 06:45:10,987 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Stopping Shard 06:45:10,987 PM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-1-shard-people-config_testRemoveAllShardReplicas: journal closed 06:45:11,16 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveAllShardReplicas: leader transfer aborted 06:45:11,16 PM [cluster-test-shard-dispatcher-18] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveAllShardReplicas: Failed to transfer leadership in 399.4 ms 06:45:11,16 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: In unpauseLeader 06:45:11,17 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Stopping Shard 06:45:11,18 PM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testRemoveAllShardReplicas: journal closed 06:45:11,126 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testRemoveAllShardReplicas: leader transfer aborted 06:45:11,126 PM [cluster-test-shard-dispatcher-19] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testRemoveAllShardReplicas: Failed to transfer leadership in 509.7 ms 06:45:11,127 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: In unpauseLeader 06:45:11,127 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Stopping Shard 06:45:11,128 PM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-1-shard-pets-config_testRemoveAllShardReplicas: journal closed 06:45:11,128 PM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 06:45:11,128 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 06:45:11,129 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 06:45:11,129 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testRemoveAllShardReplicas 06:45:11,130 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testRemoveAllShardReplicas 06:45:11,130 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 06:45:11,130 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@e5022dc 06:45:11,130 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@e5022dc 06:45:11,130 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5451a4e4 06:45:11,130 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 06:45:11,130 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4b00d0c1 06:45:11,130 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-oper_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5451a4e4 06:45:11,130 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 06:45:11,130 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6588164b 06:45:11,436 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveAllShardReplicas: leader transfer aborted 06:45:11,436 PM [cluster-test-shard-dispatcher-18] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveAllShardReplicas: Failed to transfer leadership in 306.3 ms 06:45:11,436 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: In unpauseLeader 06:45:11,437 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Stopping Shard 06:45:11,438 PM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testRemoveAllShardReplicas: journal closed 06:45:11,676 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveAllShardReplicas: leader transfer aborted 06:45:11,676 PM [cluster-test-shard-dispatcher-18] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveAllShardReplicas: Failed to transfer leadership in 546.2 ms 06:45:11,676 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: In unpauseLeader 06:45:11,677 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Stopping Shard 06:45:11,678 PM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testRemoveAllShardReplicas: journal closed 06:45:11,678 PM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 06:45:11,678 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 06:45:11,680 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:45:11,680 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:45:11,683 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/07/2025 18:45:11.686] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:45:11,713 PM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 06:45:11,723 PM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-8631947572937003667] 06:45:11,723 PM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 06:45:11,725 PM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 06:45:11,725 PM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 06:45:11,726 PM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:45:11,726 PM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:45:11,738 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:11,739 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:11,740 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 06:45:11,740 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 06:45:11,741 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 06:45:11,741 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplicaFailures 06:45:11,748 PM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#-1845765393] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:11,756 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 06:45:11,756 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 06:45:11,758 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 06:45:11,758 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testAddShardReplicaFailures 06:45:11,758 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:11,758 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures 06:45:11,762 PM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testAddShardReplicaFailures: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplicaFailures, generation=0} 06:45:11,763 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@61d0ad08} 06:45:11,763 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplicaFailures, generation=0} started 06:45:11,764 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:11,764 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:11,764 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testAddShardReplicaFailures 06:45:11,765 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures 06:45:11,765 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures to pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures 06:45:11,765 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testAddShardReplicaFailures on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures on shard member-1-shard-cars-config_testAddShardReplicaFailures whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures#-1209235559] 06:45:11,765 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:11,766 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures#-1209235559] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:11,766 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: Shard created, persistent : true 06:45:11,767 PM [cluster-test-shard-dispatcher-17] [INFO] RaftActor - member-1-shard-cars-config_testAddShardReplicaFailures: Starting recovery with journal batch size 1 06:45:11,768 PM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures-notifier#700397257 created and ready for shard:member-1-shard-cars-config_testAddShardReplicaFailures 06:45:11,768 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures-notifier#700397257] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:11,768 PM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testAddShardReplicaFailures: journal open: applyTo=0 06:45:11,768 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplicaFailures 06:45:11,770 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures#2136554867] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:11,770 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:11,770 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 06:45:11,770 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 06:45:11,770 PM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-1-shard-cars-config_testAddShardReplicaFailures: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:11,771 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 06:45:11,772 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testAddShardReplicaFailures 06:45:11,772 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:11,772 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures 06:45:11,775 PM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-1-shard-cars-config_testAddShardReplicaFailures: Local TermInfo store seeded with TermInfo{term=0} 06:45:11,775 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 06:45:11,775 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testAddShardReplicaFailures is initialized 06:45:11,775 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 06:45:11,775 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:11,775 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , received role change from null to Follower 06:45:11,776 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testAddShardReplicaFailures: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplicaFailures, generation=0} 06:45:11,777 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@24d22f56} 06:45:11,777 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplicaFailures, generation=0} started 06:45:11,777 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:11,777 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testAddShardReplicaFailures 06:45:11,777 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures 06:45:11,778 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures to pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures 06:45:11,778 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testAddShardReplicaFailures on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures on shard member-1-shard-cars-oper_testAddShardReplicaFailures whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures#361131288] 06:45:11,778 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: Shard created, persistent : true 06:45:11,778 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-oper_testAddShardReplicaFailures: Starting recovery with journal batch size 1 06:45:11,779 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=people} 06:45:11,779 PM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-cars-config_testAddShardReplicaFailures (Candidate): Starting new election term 1 06:45:11,779 PM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddShardReplicaFailures (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:45:11,779 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 06:45:11,779 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:11,779 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures#361131288] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:11,779 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , received role change from Follower to Candidate 06:45:11,779 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures-notifier#-1516964461] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:11,779 PM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddShardReplicaFailures (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 06:45:11,779 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplicaFailures: onLeadershipChange, isLeader: true, hasLeader : true 06:45:11,779 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: became leader with frontend state for [] 06:45:11,779 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , received role change from Candidate to Leader 06:45:11,779 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures-notifier#-1516964461 created and ready for shard:member-1-shard-cars-oper_testAddShardReplicaFailures 06:45:11,779 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures 06:45:11,779 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: onAddShardReplica: AddShardReplica[ShardName=people] 06:45:11,779 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: No module configuration exists for shard people 06:45:11,780 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testAddShardReplicaFailures: journal open: applyTo=0 06:45:11,780 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2f19ae64 06:45:11,780 PM [cluster-test-pekko.actor.default-dispatcher-19] [ERROR] ClusterAdminRpcService - Failed to add replica for shard ShardName{value=people} java.lang.IllegalArgumentException: No module configuration exists for shard people at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onAddShardReplica(ShardManager.java:1162) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:246) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 06:45:11,780 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:11,781 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-oper_testAddShardReplicaFailures: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:11,781 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplicaFailures, generation=0} 06:45:11,780 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddShardReplicaFailures from Candidate to Leader 06:45:11,781 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplicaFailures is ready 06:45:11,781 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testAddShardReplicaFailures 06:45:11,781 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 06:45:11,782 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@1569cc25 06:45:11,782 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplicaFailures: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@1569cc25 06:45:11,782 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: Stopping Shard 06:45:11,783 PM [cluster-test-shard-dispatcher-20] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testAddShardReplicaFailures: journal closed 06:45:11,783 PM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 06:45:11,783 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 06:45:11,783 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplicaFailures, generation=0} 06:45:11,783 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testAddShardReplicaFailures 06:45:11,784 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-oper_testAddShardReplicaFailures: Local TermInfo store seeded with TermInfo{term=0} 06:45:11,784 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 1 shards 06:45:11,784 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#-1845765393] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#-1845765393] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:11,784 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#-1845765393] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#-1845765393] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:11,784 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 06:45:11,784 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:11,784 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda/0x00007f4ea8809e48] to Actor[pekko://cluster-test/user/$a#1537776390] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/$a#1537776390] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:11,784 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , received role change from null to Follower 06:45:11,786 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-oper_testAddShardReplicaFailures (Candidate): Starting new election term 1 06:45:11,787 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddShardReplicaFailures (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:45:11,787 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 06:45:11,787 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:11,787 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , received role change from Follower to Candidate 06:45:11,787 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddShardReplicaFailures (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 06:45:11,787 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplicaFailures: onLeadershipChange, isLeader: true, hasLeader : true 06:45:11,787 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: became leader with frontend state for [] 06:45:11,787 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , received role change from Candidate to Leader 06:45:11,787 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: Stopping Shard 06:45:11,787 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testAddShardReplicaFailures: journal closed 06:45:11,787 PM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 06:45:11,789 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 06:45:11,789 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testAddShardReplicaFailures is initialized 06:45:11,789 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 06:45:11,789 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] LocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.notifications.RegisterRoleChangeListener] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures#2136554867] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures#361131288] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures#361131288] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:11,790 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:45:11,790 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:45:11,814 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 06:45:11,824 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [1882669013509440404] 06:45:11,824 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 06:45:11,826 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 06:45:11,826 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 06:45:11,827 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:45:11,827 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:45:11,847 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:11,850 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:11,852 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 06:45:11,853 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 06:45:11,853 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 06:45:11,853 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 06:45:11,857 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1689085625] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:11,860 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 06:45:11,860 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 06:45:11,862 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 06:45:11,862 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testAddShardReplica 06:45:11,862 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:11,862 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 06:45:11,864 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplica, generation=0} 06:45:11,865 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@2fcd539b} 06:45:11,865 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplica, generation=0} started 06:45:11,865 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:11,865 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testAddShardReplica 06:45:11,866 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 06:45:11,866 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 06:45:11,866 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testAddShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica on shard member-1-shard-cars-config_testAddShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#285459402] 06:45:11,866 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Shard created, persistent : true 06:45:11,866 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#285459402] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:11,867 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-notifier#998441431] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:11,867 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-config_testAddShardReplica: Starting recovery with journal batch size 1 06:45:11,867 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-notifier#998441431 created and ready for shard:member-1-shard-cars-config_testAddShardReplica 06:45:11,868 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testAddShardReplica: journal open: applyTo=0 06:45:11,869 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:11,869 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:11,872 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testAddShardReplica: Local TermInfo store seeded with TermInfo{term=0} 06:45:11,872 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 06:45:11,872 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:11,872 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testAddShardReplica is initialized 06:45:11,872 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:11,873 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , received role change from null to Follower 06:45:11,875 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-config_testAddShardReplica (Candidate): Starting new election term 1 06:45:11,875 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:45:11,875 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:11,875 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:11,875 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , received role change from Follower to Candidate 06:45:11,875 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 06:45:11,875 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 06:45:11,875 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: became leader with frontend state for [] 06:45:11,876 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , received role change from Candidate to Leader 06:45:11,876 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplica 06:45:11,876 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@22e061c4 06:45:11,876 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddShardReplica from Candidate to Leader 06:45:11,876 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 06:45:11,915 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1689085625] 06:45:11,916 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:45:11,916 PM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#285459402] 06:45:11,916 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:11,918 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:11,922 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplica 06:45:11,925 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#1565537854] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:11,925 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 06:45:11,926 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 06:45:11,927 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 06:45:11,927 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testAddShardReplica 06:45:11,928 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:11,928 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 06:45:11,928 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplica, generation=0} 06:45:11,929 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplica, generation=0} started 06:45:11,929 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@4a7bf1d3} 06:45:11,930 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:11,930 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testAddShardReplica 06:45:11,930 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 06:45:11,930 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 06:45:11,930 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testAddShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica on shard member-1-shard-cars-oper_testAddShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#371357630] 06:45:11,931 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Shard created, persistent : true 06:45:11,931 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#371357630] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:11,931 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-notifier#472906493 created and ready for shard:member-1-shard-cars-oper_testAddShardReplica 06:45:11,931 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-notifier#472906493] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:11,931 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-oper_testAddShardReplica: Starting recovery with journal batch size 1 06:45:11,932 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testAddShardReplica: journal open: applyTo=0 06:45:11,933 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:11,933 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-oper_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:11,936 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-oper_testAddShardReplica: Local TermInfo store seeded with TermInfo{term=0} 06:45:11,936 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 06:45:11,936 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:11,936 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testAddShardReplica is initialized 06:45:11,936 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:11,936 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from null to Follower 06:45:11,940 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-oper_testAddShardReplica (Candidate): Starting new election term 1 06:45:11,940 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:45:11,940 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:11,940 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:11,940 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from Follower to Candidate 06:45:11,940 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 06:45:11,940 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 06:45:11,940 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: became leader with frontend state for [] 06:45:11,940 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from Candidate to Leader 06:45:11,940 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testAddShardReplica 06:45:11,941 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4e9a787a 06:45:11,941 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddShardReplica from Candidate to Leader 06:45:11,941 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 06:45:11,980 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#1565537854] 06:45:11,980 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 06:45:11,980 PM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#371357630] 06:45:11,990 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 06:45:12,8 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-1311516256972128163] 06:45:12,8 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 06:45:12,8 PM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:45:12,8 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 06:45:12,8 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:45:12,9 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:45:12,15 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:12,26 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:12,36 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 06:45:12,44 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#919282955] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:12,44 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 06:45:12,44 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 06:45:12,47 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 06:45:12,49 PM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} 06:45:12,50 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} started 06:45:12,51 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:12,52 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@6d184ec0} 06:45:12,52 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:12,52 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:12,54 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplica 06:45:12,57 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-577366459] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:12,58 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} 06:45:12,59 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} started 06:45:12,59 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 06:45:12,59 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 06:45:12,60 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 06:45:12,60 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@63dfad13} 06:45:12,60 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:12,70 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 06:45:12,72 PM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 06:45:12,74 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:12,74 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 06:45:12,74 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:12,74 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 06:45:12,844 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 06:45:12,844 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:12,844 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 06:45:12,844 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 06:45:12,844 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:12,844 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 06:45:12,844 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 06:45:12,844 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 06:45:12,845 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 06:45:12,861 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 06:45:12,861 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 06:45:12,862 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 06:45:12,862 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica] 06:45:12,864 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 06:45:12,864 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#285459402, primaryVersion=13] 06:45:12,867 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#285459402 for shard member-2-shard-cars-config_testAddShardReplica 06:45:12,868 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: Shard created, persistent : true 06:45:12,868 PM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-968435552] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:12,869 PM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-notifier#-484656752 created and ready for shard:member-2-shard-cars-config_testAddShardReplica 06:45:12,869 PM [cluster-test-shard-dispatcher-21] [INFO] RaftActor - member-2-shard-cars-config_testAddShardReplica: Starting recovery with journal batch size 1 06:45:12,869 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-notifier#-484656752] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:12,870 PM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-config_testAddShardReplica 06:45:12,870 PM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testAddShardReplica: journal open: applyTo=0 06:45:12,871 PM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 06:45:12,871 PM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 06:45:12,872 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:12,872 PM [cluster-test-shard-dispatcher-21] [INFO] PekkoRecovery - member-2-shard-cars-config_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:12,874 PM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: snapshot is durable as of 2025-08-07T18:45:12.871077845Z 06:45:12,875 PM [cluster-test-shard-dispatcher-21] [INFO] PekkoRecovery - member-2-shard-cars-config_testAddShardReplica: Local TermInfo store seeded with TermInfo{term=0} 06:45:12,875 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 06:45:12,875 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testAddShardReplica is initialized 06:45:12,875 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:12,875 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:12,875 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddShardReplica , received role change from null to Follower 06:45:12,875 PM [cluster-test-shard-dispatcher-21] [INFO] Follower - member-2-shard-cars-config_testAddShardReplica (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 06:45:12,877 PM [cluster-test-shard-dispatcher-21] [INFO] Follower - member-2-shard-cars-config_testAddShardReplica (Follower): Snapshot received from leader: member-1-shard-cars-config_testAddShardReplica 06:45:12,878 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplica 06:45:12,878 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done false 06:45:12,878 PM [cluster-test-shard-dispatcher-21] [INFO] SnapshotManager - member-2-shard-cars-config_testAddShardReplica: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 06:45:12,878 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@f38c9d1 06:45:12,878 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testAddShardReplica from null to Follower 06:45:12,878 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 06:45:12,881 PM [cluster-test-shard-dispatcher-21] [INFO] SnapshotManager - member-2-shard-cars-config_testAddShardReplica: snapshot is durable as of 2025-08-07T18:45:12.878456700Z 06:45:12,881 PM [cluster-test-shard-dispatcher-21] [INFO] ShardSnapshotCohort - member-2-shard-cars-config_testAddShardReplica: Applying snapshot 06:45:12,881 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 06:45:12,881 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-config_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-DTCL-publisher#1181080062] with name member-2-shard-cars-config_testAddShardReplica-DTCL-publisher 06:45:12,881 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-DTCL-publisher#1181080062] 06:45:12,881 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-config_testAddShardReplica: state snapshot applied in 431.9 μs 06:45:12,881 PM [cluster-test-shard-dispatcher-21] [INFO] ShardSnapshotCohort - member-2-shard-cars-config_testAddShardReplica: Done applying snapshot 06:45:12,881 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 06:45:12,881 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-DTCL-publisher#1181080062] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:12,881 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-config_testAddShardReplica-DTCL-publisher: 117.8 μs 06:45:12,882 PM [cluster-test-shard-dispatcher-29] [INFO] AbstractLeader - member-1-shard-cars-config_testAddShardReplica (Leader): Snapshot successfully installed on follower member-2-shard-cars-config_testAddShardReplica (last chunk 1) - matchIndex set to -1, nextIndex set to 0 06:45:12,884 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1ff2646 06:45:12,885 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 06:45:12,885 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testAddShardReplica} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#285459402 06:45:12,885 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard cars 06:45:12,885 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-config_testAddShardReplica 06:45:12,885 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [cars] 06:45:12,887 PM [cluster-test-shard-dispatcher-29] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testAddShardReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]} has been successfully replicated to a majority of followers 06:45:12,889 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 06:45:12,889 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:12,889 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done true 06:45:12,889 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 06:45:12,890 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#919282955] 06:45:12,890 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 06:45:12,890 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-968435552] 06:45:12,890 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-968435552] 06:45:12,890 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-577366459] 06:45:12,890 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 06:45:12,890 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-2: Local shard cars not found - shards present: [] 06:45:12,890 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 06:45:12,890 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 06:45:12,891 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 06:45:12,891 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica] 06:45:12,892 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 06:45:12,892 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#371357630, primaryVersion=13] 06:45:12,894 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#371357630 for shard member-2-shard-cars-oper_testAddShardReplica 06:45:12,895 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: Shard created, persistent : true 06:45:12,895 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#-1797301972] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:12,896 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-notifier#-678674034 created and ready for shard:member-2-shard-cars-oper_testAddShardReplica 06:45:12,896 PM [cluster-test-shard-dispatcher-21] [INFO] RaftActor - member-2-shard-cars-oper_testAddShardReplica: Starting recovery with journal batch size 1 06:45:12,896 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-notifier#-678674034] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:12,896 PM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-oper_testAddShardReplica 06:45:12,897 PM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testAddShardReplica: journal open: applyTo=0 06:45:12,898 PM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 06:45:12,898 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:12,898 PM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 06:45:12,898 PM [cluster-test-shard-dispatcher-21] [INFO] PekkoRecovery - member-2-shard-cars-oper_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:12,902 PM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: snapshot is durable as of 2025-08-07T18:45:12.898468711Z 06:45:12,902 PM [cluster-test-shard-dispatcher-21] [INFO] PekkoRecovery - member-2-shard-cars-oper_testAddShardReplica: Local TermInfo store seeded with TermInfo{term=0} 06:45:12,902 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 06:45:12,902 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testAddShardReplica is initialized 06:45:12,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:12,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:12,902 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddShardReplica , received role change from null to Follower 06:45:12,902 PM [cluster-test-shard-dispatcher-21] [INFO] Follower - member-2-shard-cars-oper_testAddShardReplica (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 06:45:12,904 PM [cluster-test-shard-dispatcher-21] [INFO] Follower - member-2-shard-cars-oper_testAddShardReplica (Follower): Snapshot received from leader: member-1-shard-cars-oper_testAddShardReplica 06:45:12,904 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testAddShardReplica 06:45:12,904 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done false 06:45:12,904 PM [cluster-test-shard-dispatcher-21] [INFO] SnapshotManager - member-2-shard-cars-oper_testAddShardReplica: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 06:45:12,905 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5c0a2a1 06:45:12,905 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testAddShardReplica from null to Follower 06:45:12,905 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 06:45:12,908 PM [cluster-test-shard-dispatcher-21] [INFO] SnapshotManager - member-2-shard-cars-oper_testAddShardReplica: snapshot is durable as of 2025-08-07T18:45:12.905346276Z 06:45:12,908 PM [cluster-test-shard-dispatcher-21] [INFO] ShardSnapshotCohort - member-2-shard-cars-oper_testAddShardReplica: Applying snapshot 06:45:12,908 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 06:45:12,909 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-oper_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher#14521537] with name member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher 06:45:12,909 PM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher#14521537] 06:45:12,909 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testAddShardReplica: state snapshot applied in 425.5 μs 06:45:12,909 PM [cluster-test-shard-dispatcher-21] [INFO] ShardSnapshotCohort - member-2-shard-cars-oper_testAddShardReplica: Done applying snapshot 06:45:12,909 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher#14521537] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:12,909 PM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 06:45:12,909 PM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher: 116.4 μs 06:45:12,910 PM [cluster-test-shard-dispatcher-29] [INFO] AbstractLeader - member-1-shard-cars-oper_testAddShardReplica (Leader): Snapshot successfully installed on follower member-2-shard-cars-oper_testAddShardReplica (last chunk 1) - matchIndex set to -1, nextIndex set to 0 06:45:12,912 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testAddShardReplica} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#371357630 06:45:12,913 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard cars 06:45:12,913 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-oper_testAddShardReplica 06:45:12,913 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars] 06:45:12,916 PM [cluster-test-shard-dispatcher-29] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testAddShardReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]} has been successfully replicated to a majority of followers 06:45:12,917 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@40c22c7d 06:45:12,917 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 06:45:12,917 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done true 06:45:12,917 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 06:45:12,917 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 06:45:12,917 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-577366459] 06:45:12,917 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 06:45:12,917 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:12,917 PM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#-1797301972] 06:45:12,917 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#-1797301972] 06:45:12,927 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 06:45:12,937 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-3996234640021150647] 06:45:12,938 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 06:45:12,940 PM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:45:12,940 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 06:45:12,945 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:45:12,946 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:45:12,951 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:12,952 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:12,956 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 06:45:12,968 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-622854769] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:12,976 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} 06:45:12,978 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 06:45:12,978 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 06:45:12,982 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} started 06:45:12,983 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:12,983 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 06:45:12,983 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@1a083a08} 06:45:12,983 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:12,986 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:12,989 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplica 06:45:12,992 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 06:45:12,992 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 06:45:12,992 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-1231982158] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:12,994 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 06:45:13,4 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} 06:45:13,5 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@14cfc739} 06:45:13,6 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} started 06:45:13,30 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:13,42 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:13,42 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 06:45:13,42 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 06:45:13,43 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:13,43 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 06:45:13,43 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 06:45:13,44 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 06:45:13,44 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 06:45:13,53 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 06:45:13,56 PM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 06:45:13,56 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:13,56 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 06:45:13,56 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:13,56 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 06:45:13,56 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:13,56 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 06:45:13,56 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:13,56 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 06:45:13,507 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done false 06:45:13,534 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done false 06:45:13,825 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done true 06:45:13,854 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done true 06:45:14,883 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 06:45:14,884 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:14,884 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 06:45:14,884 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 06:45:14,884 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 06:45:14,884 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:14,885 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 06:45:14,885 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 06:45:14,885 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 06:45:15,902 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 06:45:15,906 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:15,906 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 06:45:15,906 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:15,906 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 06:45:16,4 PM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 06:45:16,7 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:16,7 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 06:45:16,7 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 06:45:16,7 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 06:45:16,7 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:16,7 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 06:45:16,8 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 06:45:16,8 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 06:45:16,44 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 06:45:16,44 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 06:45:16,46 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 06:45:16,46 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica, visitedAddresses: [pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica] 06:45:16,48 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 06:45:16,48 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#285459402, primaryVersion=13] 06:45:16,52 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Shard created, persistent : true 06:45:16,52 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#285459402 for shard member-3-shard-cars-config_testAddShardReplica 06:45:16,52 PM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#-1851754443] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:16,53 PM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#1574788155 created and ready for shard:member-3-shard-cars-config_testAddShardReplica 06:45:16,53 PM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#1574788155] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:16,53 PM [cluster-test-shard-dispatcher-14] [INFO] RaftActor - member-3-shard-cars-config_testAddShardReplica: Starting recovery with journal batch size 1 06:45:16,54 PM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-config_testAddShardReplica 06:45:16,55 PM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: Persising snapshot at EntryInfo[index=0, term=1]/EntryInfo[index=0, term=1] 06:45:16,55 PM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 06:45:16,56 PM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testAddShardReplica: journal open: applyTo=0 06:45:16,58 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:16,58 PM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-3-shard-cars-config_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:16,62 PM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-3-shard-cars-config_testAddShardReplica: Local TermInfo store seeded with TermInfo{term=0} 06:45:16,62 PM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: snapshot is durable as of 2025-08-07T18:45:16.055771645Z 06:45:16,62 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:16,62 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:16,62 PM [cluster-test-shard-dispatcher-14] [INFO] Follower - member-3-shard-cars-config_testAddShardReplica (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1], votingConfig=VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]}}" message is greater than follower's term 0 - updating term 06:45:16,62 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 06:45:16,62 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized 06:45:16,62 PM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , received role change from null to Follower 06:45:16,64 PM [cluster-test-shard-dispatcher-14] [INFO] Follower - member-3-shard-cars-config_testAddShardReplica (Follower): Snapshot received from leader: member-1-shard-cars-config_testAddShardReplica 06:45:16,65 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done false 06:45:16,65 PM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplica 06:45:16,65 PM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-3-shard-cars-config_testAddShardReplica: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 06:45:16,65 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@10f05178 06:45:16,65 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testAddShardReplica from null to Follower 06:45:16,65 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 06:45:16,68 PM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-3-shard-cars-config_testAddShardReplica: snapshot is durable as of 2025-08-07T18:45:16.065482205Z 06:45:16,68 PM [cluster-test-shard-dispatcher-12] [INFO] ShardSnapshotCohort - member-3-shard-cars-config_testAddShardReplica: Applying snapshot 06:45:16,68 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 06:45:16,69 PM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#-427289502] 06:45:16,69 PM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#-427289502] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:16,69 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-config_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#-427289502] with name member-3-shard-cars-config_testAddShardReplica-DTCL-publisher 06:45:16,69 PM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 06:45:16,69 PM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-config_testAddShardReplica-DTCL-publisher: 122.8 μs 06:45:16,69 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-3-shard-cars-config_testAddShardReplica: state snapshot applied in 1.489 ms 06:45:16,69 PM [cluster-test-shard-dispatcher-12] [INFO] ShardSnapshotCohort - member-3-shard-cars-config_testAddShardReplica: Done applying snapshot 06:45:16,70 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done true 06:45:16,70 PM [cluster-test-shard-dispatcher-11] [INFO] AbstractLeader - member-1-shard-cars-config_testAddShardReplica (Leader): Snapshot successfully installed on follower member-3-shard-cars-config_testAddShardReplica (last chunk 1) - matchIndex set to 0, nextIndex set to 1 06:45:16,73 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testAddShardReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]} has been successfully replicated to a majority of followers 06:45:16,73 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testAddShardReplica} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#285459402 06:45:16,74 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully added the replica shard cars 06:45:16,75 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-config_testAddShardReplica 06:45:16,75 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [cars] 06:45:16,77 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 06:45:16,77 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:16,79 PM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 06:45:16,79 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-622854769] 06:45:16,79 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 06:45:16,79 PM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#-1851754443] 06:45:16,79 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#-1851754443] 06:45:16,80 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-1231982158] 06:45:16,80 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 06:45:16,80 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: Local shard cars not found - shards present: [] 06:45:16,80 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 06:45:16,80 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 06:45:16,81 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 06:45:16,81 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica, visitedAddresses: [pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica] 06:45:16,82 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 06:45:16,82 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#371357630, primaryVersion=13] 06:45:16,83 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#371357630 for shard member-3-shard-cars-oper_testAddShardReplica 06:45:16,84 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: Shard created, persistent : true 06:45:16,84 PM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#-877563025] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:16,85 PM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-oper_testAddShardReplica 06:45:16,85 PM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: Persising snapshot at EntryInfo[index=0, term=1]/EntryInfo[index=0, term=1] 06:45:16,85 PM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 06:45:16,86 PM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-notifier#1244509892 created and ready for shard:member-3-shard-cars-oper_testAddShardReplica 06:45:16,86 PM [cluster-test-shard-dispatcher-28] [INFO] RaftActor - member-3-shard-cars-oper_testAddShardReplica: Starting recovery with journal batch size 1 06:45:16,86 PM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-notifier#1244509892] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:16,89 PM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: snapshot is durable as of 2025-08-07T18:45:16.085770800Z 06:45:16,89 PM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testAddShardReplica: journal open: applyTo=0 06:45:16,90 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:16,91 PM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-cars-oper_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:16,93 PM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-cars-oper_testAddShardReplica: Local TermInfo store seeded with TermInfo{term=0} 06:45:16,94 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 06:45:16,94 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testAddShardReplica is initialized 06:45:16,94 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:16,94 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:16,94 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testAddShardReplica , received role change from null to Follower 06:45:16,94 PM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-3-shard-cars-oper_testAddShardReplica (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1], votingConfig=VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]}}" message is greater than follower's term 0 - updating term 06:45:16,96 PM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-3-shard-cars-oper_testAddShardReplica (Follower): Snapshot received from leader: member-1-shard-cars-oper_testAddShardReplica 06:45:16,97 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done false 06:45:16,97 PM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testAddShardReplica 06:45:16,97 PM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-3-shard-cars-oper_testAddShardReplica: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 06:45:16,97 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@88e59b5 06:45:16,97 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testAddShardReplica from null to Follower 06:45:16,97 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 06:45:16,101 PM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-3-shard-cars-oper_testAddShardReplica: snapshot is durable as of 2025-08-07T18:45:16.097593910Z 06:45:16,101 PM [cluster-test-shard-dispatcher-28] [INFO] ShardSnapshotCohort - member-3-shard-cars-oper_testAddShardReplica: Applying snapshot 06:45:16,101 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-3-shard-cars-oper_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 06:45:16,102 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-oper_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher#-682678728] with name member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher 06:45:16,102 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testAddShardReplica: state snapshot applied in 542.3 μs 06:45:16,102 PM [cluster-test-shard-dispatcher-28] [INFO] ShardSnapshotCohort - member-3-shard-cars-oper_testAddShardReplica: Done applying snapshot 06:45:16,102 PM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher#-682678728] 06:45:16,102 PM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 06:45:16,102 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher#-682678728] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:16,102 PM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher: 126.6 μs 06:45:16,103 PM [cluster-test-shard-dispatcher-11] [INFO] AbstractLeader - member-1-shard-cars-oper_testAddShardReplica (Leader): Snapshot successfully installed on follower member-3-shard-cars-oper_testAddShardReplica (last chunk 1) - matchIndex set to 0, nextIndex set to 1 06:45:16,103 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done true 06:45:16,106 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testAddShardReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]} has been successfully replicated to a majority of followers 06:45:16,106 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testAddShardReplica} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#371357630 06:45:16,106 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully added the replica shard cars 06:45:16,106 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-oper_testAddShardReplica 06:45:16,106 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [cars] 06:45:16,109 PM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 06:45:16,109 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-1231982158] 06:45:16,109 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 06:45:16,109 PM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#-877563025] 06:45:16,109 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#-877563025] 06:45:16,112 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#919282955] 06:45:16,112 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 06:45:16,112 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:16,112 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 06:45:16,112 PM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-968435552] 06:45:16,112 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-968435552] 06:45:16,113 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-577366459] 06:45:16,113 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 06:45:16,113 PM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#-1797301972] 06:45:16,113 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#-1797301972] 06:45:16,155 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#919282955] 06:45:16,155 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 06:45:16,155 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-2: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 06:45:16,156 PM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 06:45:16,170 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} is not yet known 06:45:16,179 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, nanosAgo=55001, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} 06:45:16,185 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#285459402], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 06:45:16,187 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#285459402], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 06:45:16,198 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#285459402], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 11.40 ms 06:45:16,222 PM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: allocating new open transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 06:45:16,241 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: readying transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 06:45:16,241 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 ready 06:45:16,245 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: transitioned member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 to ready 06:45:16,245 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 initiating direct canCommit 06:45:16,246 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Validating transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 06:45:16,246 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 validated 06:45:16,246 PM [cluster-test-shard-dispatcher-11] [DEBUG] CommitCohort - Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 06:45:16,246 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 initiating direct preCommit 06:45:16,246 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Preparing transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 06:45:16,249 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 candidate ready 06:45:16,250 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: canCommit - candidate rootPath: / 06:45:16,254 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: canCommit - messages: [] 06:45:16,254 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 06:45:16,254 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 prepared 06:45:16,254 PM [cluster-test-shard-dispatcher-11] [DEBUG] CommitCohort - Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 06:45:16,254 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 initiating direct commit 06:45:16,255 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Starting commit for transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 06:45:16,259 PM [cluster-test-shard-dispatcher-11] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 235 06:45:16,261 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 submitted to persistence 06:45:16,267 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done false 06:45:16,267 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Resuming commit of transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 06:45:16,269 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-1-shard-cars-config_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@60b57ffb}, purged=MutableUnsignedLongSet{size=0}} 06:45:16,269 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 06:45:16,271 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 06:45:16,271 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: commit - successfulFromPrevious: [] 06:45:16,271 PM [cluster-test-shard-dispatcher-11] [DEBUG] CommitCohort - Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 06:45:16,272 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-config_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-DTCL-publisher#1448738632] with name member-1-shard-cars-config_testAddShardReplica-DTCL-publisher 06:45:16,273 PM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-DTCL-publisher#1448738632] 06:45:16,273 PM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 06:45:16,273 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-DTCL-publisher#1448738632] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:16,273 PM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-1-shard-cars-config_testAddShardReplica-DTCL-publisher: 128.4 μs 06:45:16,280 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-622854769] 06:45:16,280 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 06:45:16,281 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 06:45:16,281 PM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 06:45:16,288 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-cars-config_testAddShardReplica: Applying foreign transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 06:45:16,288 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-3-shard-cars-config_testAddShardReplica: Applying foreign transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 06:45:16,288 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: purging transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 06:45:16,290 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@10fbe6af}, purged=MutableUnsignedLongSet{size=0}} 06:45:16,290 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 06:45:16,290 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 06:45:16,290 PM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 06:45:16,290 PM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-config_testAddShardReplica-DTCL-publisher: 105.0 μs 06:45:16,290 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@bcbb812}, purged=MutableUnsignedLongSet{size=0}} 06:45:16,290 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 06:45:16,290 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 06:45:16,290 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done true 06:45:16,291 PM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 06:45:16,291 PM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-config_testAddShardReplica-DTCL-publisher: 84.22 μs 06:45:16,295 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} is not yet known 06:45:16,295 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, nanosAgo=12190, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} 06:45:16,296 PM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#285459402], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 06:45:16,296 PM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#285459402], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 06:45:16,297 PM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#285459402], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 607.6 μs 06:45:16,297 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: replication of member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda/0x00007f4ea8976210@675e7e7d 06:45:16,299 PM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: finished purging transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 06:45:16,299 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 06:45:16,299 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 06:45:16,302 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 06:45:16,302 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 06:45:16,303 PM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 06:45:16,303 PM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 06:45:16,305 PM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: allocating new open snapshot member-3-datastore-config_testAddShardReplica-fe-0-txn-0-1 06:45:16,312 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-1231982158] 06:45:16,312 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 06:45:16,313 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 06:45:16,313 PM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 06:45:16,314 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} is not yet known 06:45:16,314 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, nanosAgo=15031, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} 06:45:16,315 PM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#371357630], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 06:45:16,315 PM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#371357630], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 06:45:16,317 PM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#371357630], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.884 ms 06:45:16,318 PM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: allocating new open transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 06:45:16,318 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: readying transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 06:45:16,318 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 ready 06:45:16,318 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: transitioned member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 to ready 06:45:16,318 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 initiating direct canCommit 06:45:16,318 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Validating transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 06:45:16,318 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 validated 06:45:16,318 PM [cluster-test-shard-dispatcher-11] [DEBUG] CommitCohort - Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 06:45:16,318 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 initiating direct preCommit 06:45:16,319 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Preparing transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 06:45:16,319 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 candidate ready 06:45:16,319 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: canCommit - candidate rootPath: / 06:45:16,319 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: canCommit - messages: [] 06:45:16,319 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 06:45:16,319 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 prepared 06:45:16,319 PM [cluster-test-shard-dispatcher-11] [DEBUG] CommitCohort - Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 06:45:16,319 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 initiating direct commit 06:45:16,319 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Starting commit for transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 06:45:16,319 PM [cluster-test-shard-dispatcher-11] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 233 06:45:16,320 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 submitted to persistence 06:45:16,322 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done false 06:45:16,322 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Resuming commit of transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 06:45:16,322 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-1-shard-cars-oper_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@49b14bed}, purged=MutableUnsignedLongSet{size=0}} 06:45:16,322 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 06:45:16,322 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: Committed transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 06:45:16,322 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: commit - successfulFromPrevious: [] 06:45:16,322 PM [cluster-test-shard-dispatcher-11] [DEBUG] CommitCohort - Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 06:45:16,322 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-oper_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher#-1844764121] with name member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher 06:45:16,322 PM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher#-1844764121] 06:45:16,322 PM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 06:45:16,323 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher#-1844764121] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:16,323 PM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher: 119.7 μs 06:45:16,324 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-577366459] 06:45:16,324 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 06:45:16,324 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-2: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 06:45:16,324 PM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 06:45:16,325 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: purging transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 06:45:16,326 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testAddShardReplica: Applying foreign transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 06:45:16,326 PM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendMetadata - member-3-shard-cars-oper_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@18eacf10}, purged=MutableUnsignedLongSet{size=0}} 06:45:16,326 PM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 06:45:16,326 PM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: Committed transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 06:45:16,326 PM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 06:45:16,326 PM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher: 83.62 μs 06:45:16,326 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done true 06:45:16,327 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} is not yet known 06:45:16,327 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, nanosAgo=9830, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} 06:45:16,327 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testAddShardReplica: Applying foreign transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 06:45:16,327 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@67409f83}, purged=MutableUnsignedLongSet{size=0}} 06:45:16,327 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 06:45:16,327 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: Committed transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 06:45:16,327 PM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 06:45:16,327 PM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher: 92.50 μs 06:45:16,328 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#371357630], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 06:45:16,328 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#371357630], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 06:45:16,328 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#371357630], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 312.2 μs 06:45:16,329 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: replication of member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda/0x00007f4ea8976210@1e452341 06:45:16,329 PM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: finished purging transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 06:45:16,329 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 06:45:16,329 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: Purged transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 06:45:16,329 PM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: allocating new open snapshot member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 06:45:16,330 PM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 06:45:16,330 PM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: Purged transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 06:45:16,332 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1689085625] 06:45:16,332 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 06:45:16,332 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: Purged transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 06:45:16,332 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:45:16,332 PM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#285459402] 06:45:16,332 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#285459402] 06:45:16,332 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#1565537854] 06:45:16,332 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 06:45:16,332 PM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#371357630] 06:45:16,332 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#371357630] 06:45:16,333 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#919282955] 06:45:16,333 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 06:45:16,333 PM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-968435552] 06:45:16,333 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-968435552] 06:45:16,333 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-577366459] 06:45:16,333 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 06:45:16,333 PM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#-1797301972] 06:45:16,333 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#-1797301972] 06:45:16,333 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-622854769] 06:45:16,333 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 06:45:16,333 PM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#-1851754443] 06:45:16,333 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#-1851754443] 06:45:16,333 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-1231982158] 06:45:16,333 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 06:45:16,333 PM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#-877563025] 06:45:16,334 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#-877563025] 06:45:16,334 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} 06:45:16,334 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testAddShardReplica 06:45:16,334 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 06:45:16,334 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node [pekko://cluster-test@127.0.0.1:2557] as [Down] 06:45:16,335 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Stopping Shard 06:45:16,335 PM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testAddShardReplica: journal closed 06:45:16,335 PM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 06:45:16,335 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 06:45:16,336 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} 06:45:16,336 PM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-622854769] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-622854769] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:16,337 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testAddShardReplica 06:45:16,337 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 1 shards 06:45:16,337 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: Stopping Shard 06:45:16,337 PM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-622854769] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-622854769] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:16,337 PM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testAddShardReplica: journal closed 06:45:16,337 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 06:45:16,337 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 06:45:16,338 PM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Received gossip where this member has been downed, from [pekko://cluster-test@127.0.0.1:2558] 06:45:16,338 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Node has been marked as DOWN. Shutting down myself 06:45:16,338 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Shutting down... 06:45:16,340 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Successfully shut down 06:45:16,340 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] CoordinatedShutdown - Running CoordinatedShutdown with reason [ClusterDowningReason] 06:45:16,341 PM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:45:16,341 PM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:45:16,347 PM [cluster-test-pekko.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:16,347 PM [cluster-test-pekko.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:16,348 PM [cluster-test-pekko.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:16,348 PM [cluster-test-pekko.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:16,349 PM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 06:45:16,374 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 06:45:16,382 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [8758152854228188180] 06:45:16,382 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 06:45:16,384 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Registered cluster JMX MBean [pekko:type=Cluster] 06:45:16,384 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 06:45:16,386 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:45:16,386 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:45:16,395 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:16,397 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:16,401 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 06:45:16,408 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#46204148] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:16,409 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 06:45:16,409 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: applying snapshot ShardManagerSnapshot [ShardList = [cars] ] 06:45:16,409 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: adding shard cars 06:45:16,413 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 06:45:16,413 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testAddShardReplica 06:45:16,414 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@1da69445} 06:45:16,415 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} started 06:45:16,415 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:16,415 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testAddShardReplica 06:45:16,415 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#46204148] 06:45:16,416 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 06:45:16,416 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 300000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-cars-config_testAddShardReplica, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 06:45:16,417 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Shard created, persistent : true 06:45:16,417 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#2020048688] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:16,417 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-3-shard-cars-config_testAddShardReplica: Starting recovery with journal batch size 1 06:45:16,418 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#956059994 created and ready for shard:member-3-shard-cars-config_testAddShardReplica 06:45:16,418 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#956059994] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:16,430 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testAddShardReplica: journal open: applyTo=3 06:45:16,431 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:16,433 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardRecoveryCoordinator - member-3-shard-cars-config_testAddShardReplica: Applying recovered snapshot 06:45:16,438 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 06:45:16,441 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-config_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#-267394129] with name member-3-shard-cars-config_testAddShardReplica-DTCL-publisher 06:45:16,441 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#-267394129] 06:45:16,441 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-3-shard-cars-config_testAddShardReplica: state snapshot applied in 3.298 ms 06:45:16,441 PM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - Recovery snapshot applied for member-3-shard-cars-config_testAddShardReplica in 8.177 ms: snapshotIndex=0, snapshotTerm=1, journal-size=0 06:45:16,441 PM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-3-shard-cars-config_testAddShardReplica: Recovery completed in 8.301 ms - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = 0, snapshot term = 1, journal size = 0 06:45:16,441 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 06:45:16,441 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-config_testAddShardReplica-DTCL-publisher: 100.6 μs 06:45:16,442 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#-267394129] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:16,443 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardRecoveryCoordinator - member-3-shard-cars-config_testAddShardReplica: starting log recovery batch with max size 1 06:45:16,449 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@79b9eea7}, purged=MutableUnsignedLongSet{size=0}} 06:45:16,449 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 06:45:16,449 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 06:45:16,451 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardRecoveryCoordinator - member-3-shard-cars-config_testAddShardReplica: starting log recovery batch with max size 1 06:45:16,451 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 06:45:16,451 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 06:45:16,451 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 06:45:16,451 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:16,451 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized 06:45:16,451 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:16,451 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized - notifying 1 OnShardInitialized callbacks 06:45:16,452 PM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#2020048688] 06:45:16,452 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#2020048688] 06:45:16,452 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , received role change from null to Follower 06:45:16,452 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#46204148] 06:45:16,452 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 06:45:16,452 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 18000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-cars-config_testAddShardReplica, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 06:45:16,453 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplica 06:45:16,453 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testAddShardReplica from null to Follower 06:45:16,453 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized - notifying 1 OnShardInitialized callbacks 06:45:16,923 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 06:45:17,355 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-3996234640021150647] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 06:45:17,362 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2557] ignored due to non-matching UID, quarantine requested for [-3996234640021150647] but current is [8758152854228188180]. Reason: ActorSystem terminated 06:45:17,381 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 06:45:17,401 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 06:45:17,438 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#371357630] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:17,447 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#371357630] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:17,447 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#371357630] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:17,447 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7b0bf02b 06:45:17,447 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized - notifying 1 OnShardInitialized callbacks 06:45:17,448 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done false 06:45:17,448 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done true 06:45:17,450 PM [Cleaner-0] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-3-datastore-config_testAddShardReplica-fe-0-txn-0-0} 06:45:17,451 PM [Cleaner-0] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-0} 06:45:17,451 PM [Cleaner-0] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-3-datastore-config_testAddShardReplica-fe-1-txn-0-0} 06:45:17,452 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: aborting transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 06:45:17,694 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#371357630] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:17,695 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: replication of member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.FrontendReadOnlyTransaction$$Lambda/0x00007f4ea8982020@5ee72931 06:45:17,696 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-1-shard-cars-oper_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@426e9063}, purged=MutableUnsignedLongSet{size=0}} 06:45:17,696 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 06:45:17,696 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: Aborted transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 06:45:17,700 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@16c7e0fa}, purged=MutableUnsignedLongSet{size=0}} 06:45:17,700 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 06:45:17,700 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: Aborted transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 06:45:17,701 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: purging transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 06:45:17,983 PM [cluster-test-shard-dispatcher-22] [INFO] PeerInfos - member-3-shard-cars-config_testAddShardReplica: Peer address for peer member-1-shard-cars-config_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 06:45:18,14 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#371357630] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:18,15 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: replication of member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda/0x00007f4ea8976210@66281aea 06:45:18,15 PM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: finished purging transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 06:45:18,15 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 06:45:18,15 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: Purged transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 06:45:18,17 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 06:45:18,17 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: Purged transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 06:45:18,78 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2559/system/clusterReceptionist/replicator#-120267572]] intended for system uid [-3996234640021150647], self uid is [8758152854228188180] 06:45:18,334 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#371357630] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:18,422 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 06:45:19,443 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 06:45:19,923 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2558/system/clusterReceptionist/replicator#342569160]] intended for system uid [-3996234640021150647], self uid is [8758152854228188180] 06:45:20,98 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2559/system/clusterReceptionist/replicator#-120267572]] intended for system uid [-3996234640021150647], self uid is [8758152854228188180] 06:45:20,463 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 06:45:21,3 PM [cluster-test-pekko.actor.default-dispatcher-13] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Down)]. 06:45:21,3 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:21,3 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:21,186 PM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Down)]. 06:45:21,187 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:21,187 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:21,482 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 06:45:21,943 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2558/system/clusterReceptionist/replicator#342569160]] intended for system uid [-3996234640021150647], self uid is [8758152854228188180] 06:45:22,25 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is removing unreachable node [pekko://cluster-test@127.0.0.1:2557] 06:45:22,26 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberRemoved: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:22,26 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberRemoved: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:22,27 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2557] ignored due to non-matching UID, quarantine requested for [-3996234640021150647] but current is [8758152854228188180]. Reason: Cluster member removed, previous status [Down] 06:45:22,117 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2559/system/clusterReceptionist/replicator#-120267572]] intended for system uid [-3996234640021150647], self uid is [8758152854228188180] 06:45:22,216 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberRemoved: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:22,216 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received MemberRemoved: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:22,217 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2557] ignored due to non-matching UID, quarantine requested for [-3996234640021150647] but current is [8758152854228188180]. Reason: Cluster member removed, previous status [Down] 06:45:22,502 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 06:45:22,506 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 06:45:22,508 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:22,508 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 06:45:22,508 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 06:45:22,508 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is ready - notifying 1 OnShardInitialized callbacks 06:45:22,508 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 06:45:22,509 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 06:45:22,509 PM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 06:45:22,509 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:22,509 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 06:45:22,509 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 06:45:22,509 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 06:45:22,509 PM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-3-shard-cars-config_testAddShardReplica: Peer address for peer member-1-shard-cars-config_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 06:45:22,509 PM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-3-shard-cars-config_testAddShardReplica: Peer address for peer member-2-shard-cars-config_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 06:45:22,510 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: retiring state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, nanosAgo=6205141646, purgedHistories=MutableUnsignedLongSet{size=0}}, outdated by request from client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} 06:45:22,512 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, nanosAgo=25941, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} 06:45:22,513 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#285459402], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 06:45:22,513 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#285459402], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 06:45:22,514 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#285459402], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.055 ms 06:45:22,515 PM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: allocating new open snapshot member-3-datastore-config_testAddShardReplica-fe-1-txn-0-1 06:45:22,515 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: aborting transaction member-3-datastore-config_testAddShardReplica-fe-1-txn-0-1 06:45:22,517 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} 06:45:22,518 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testAddShardReplica 06:45:22,518 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 06:45:22,518 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Stopping Shard 06:45:22,518 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testAddShardReplica: journal closed 06:45:22,519 PM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 06:45:22,519 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 06:45:22,520 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:45:22,520 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:45:22,524 PM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:22,525 PM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:22,525 PM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:22,525 PM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:22,526 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 06:45:22,541 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} 06:45:22,541 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testAddShardReplica 06:45:22,541 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 1 shards 06:45:22,542 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: Stopping Shard 06:45:22,542 PM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testAddShardReplica: journal closed 06:45:22,543 PM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 06:45:22,543 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 06:45:22,543 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} 06:45:22,544 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#919282955] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#919282955] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:22,544 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#919282955] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#919282955] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:22,544 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testAddShardReplica 06:45:22,544 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 1 shards 06:45:22,544 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: Stopping Shard 06:45:22,545 PM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testAddShardReplica: journal closed 06:45:22,545 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 06:45:22,545 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 06:45:22,546 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:45:22,546 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:45:23,543 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [8758152854228188180] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 06:45:23,562 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-1311516256972128163] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 06:45:23,567 PM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:23,568 PM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:23,570 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/07/2025 18:45:23.572] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:45:23,576 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplica, generation=0} 06:45:23,577 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testAddShardReplica 06:45:23,577 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 06:45:23,577 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7625ecca 06:45:23,577 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@18814aae 06:45:23,578 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@18814aae 06:45:23,578 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddShardReplica: pauseLeader successfully completed - doing transfer 06:45:24,63 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 06:45:24,64 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:24,64 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 06:45:24,64 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 06:45:24,64 PM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-1-shard-cars-config_testAddShardReplica: Peer address for peer member-3-shard-cars-config_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 06:45:24,64 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:24,64 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 06:45:24,64 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 06:45:24,64 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 06:45:24,64 PM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-1-shard-cars-oper_testAddShardReplica: Peer address for peer member-3-shard-cars-oper_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 06:45:25,82 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 06:45:28,143 PM [cluster-test-pekko.actor.default-dispatcher-13] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2559, Up)]. 06:45:28,143 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:28,144 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:29,953 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#285459402] to Actor[pekko://cluster-test/deadLetters] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:29,953 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#371357630] to Actor[pekko://cluster-test/deadLetters] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:30,149 PM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 06:45:30,182 PM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Up)]. 06:45:30,182 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:30,182 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:31,109 PM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 06:45:31,203 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 06:45:32,382 PM [cluster-test-shard-dispatcher-29] [WARN] Leader - member-1-shard-cars-config_testAddShardReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-config_testAddShardReplica from Leader to IsolatedLeader 06:45:32,382 PM [cluster-test-shard-dispatcher-29] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 06:45:32,382 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddShardReplica: leader transfer aborted 06:45:32,382 PM [cluster-test-shard-dispatcher-29] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddShardReplica: Failed to transfer leadership in 8.805 s 06:45:32,382 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: In unpauseLeader 06:45:32,385 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 06:45:32,385 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , received role change from Leader to IsolatedLeader 06:45:32,385 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddShardReplica from Leader to IsolatedLeader 06:45:32,385 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Stopping Shard 06:45:32,386 PM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testAddShardReplica: journal closed 06:45:32,386 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 06:45:32,386 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 06:45:32,387 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplica, generation=0} 06:45:32,387 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testAddShardReplica 06:45:32,388 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 1 shards 06:45:32,388 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@38d43963 06:45:32,388 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@30139abe 06:45:32,388 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@30139abe 06:45:32,388 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testAddShardReplica: pauseLeader successfully completed - doing transfer 06:45:32,412 PM [cluster-test-shard-dispatcher-29] [WARN] Leader - member-1-shard-cars-oper_testAddShardReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-oper_testAddShardReplica from Leader to IsolatedLeader 06:45:32,413 PM [cluster-test-shard-dispatcher-29] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 06:45:32,413 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testAddShardReplica: leader transfer aborted 06:45:32,413 PM [cluster-test-shard-dispatcher-29] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testAddShardReplica: Failed to transfer leadership in 24.80 ms 06:45:32,413 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: In unpauseLeader 06:45:32,413 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 06:45:32,413 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from Leader to IsolatedLeader 06:45:32,413 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddShardReplica from Leader to IsolatedLeader 06:45:32,413 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Stopping Shard 06:45:32,414 PM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testAddShardReplica: journal closed 06:45:32,414 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 06:45:32,414 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 06:45:32,416 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:45:32,416 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:45:32,420 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/07/2025 18:45:32.423] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:45:32,448 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 06:45:32,458 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [8191223470072820212] 06:45:32,459 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 06:45:32,461 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 06:45:32,461 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 06:45:32,462 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:45:32,463 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:45:32,472 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:32,474 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:32,476 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForAllShards 06:45:32,480 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 06:45:32,480 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 06:45:32,480 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 06:45:32,480 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 06:45:32,481 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 06:45:32,481 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1829486303] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:32,482 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 06:45:32,482 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:32,483 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:32,483 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:32,483 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:32,483 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:32,489 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 06:45:32,490 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@36edde6c} 06:45:32,490 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} started 06:45:32,490 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:32,491 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:32,491 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:32,491 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:32,491 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:32,491 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards on shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#-1806804083] 06:45:32,491 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:32,491 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:32,492 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:32,492 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards on shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#-1270573800] 06:45:32,492 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 06:45:32,492 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#-1806804083] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:32,492 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:32,493 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#974583891 created and ready for shard:member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:32,493 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#974583891] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:32,493 PM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 06:45:32,494 PM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 06:45:32,495 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForAllShards 06:45:32,495 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 06:45:32,495 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#-1270573800] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:32,496 PM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 06:45:32,496 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:32,496 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:32,496 PM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 06:45:32,496 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#-1191035619] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:32,498 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 06:45:32,499 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 06:45:32,499 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:32,499 PM [cluster-test-shard-dispatcher-13] [INFO] PekkoRecovery - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:32,501 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 06:45:32,501 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 06:45:32,501 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:32,501 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 06:45:32,501 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 06:45:32,501 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards is initialized 06:45:32,503 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 06:45:32,504 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#-1191035619 created and ready for shard:member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:32,504 PM [cluster-test-shard-dispatcher-13] [INFO] PekkoRecovery - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 06:45:32,504 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 06:45:32,504 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:32,505 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 06:45:32,505 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:32,505 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 06:45:32,505 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#1185558638] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:32,505 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:32,505 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:32,505 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:32,505 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:32,505 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards from null to Follower 06:45:32,506 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 06:45:32,506 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards is initialized 06:45:32,506 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 06:45:32,506 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards from null to Follower 06:45:32,508 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 06:45:32,508 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@7903adfc} 06:45:32,509 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} started 06:45:32,512 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:32,512 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:32,512 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:32,512 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:32,512 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-1279939738] 06:45:32,512 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:32,513 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:32,513 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:32,513 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-1279939738] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:32,513 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#-1704013988] 06:45:32,513 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#-1704013988] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:32,513 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 06:45:32,513 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#-262411269 created and ready for shard:member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:32,513 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#-262411269] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:32,513 PM [cluster-test-shard-dispatcher-20] [INFO] RaftActor - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 06:45:32,513 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 06:45:32,514 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#-142337208 created and ready for shard:member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:32,514 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#-142337208] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:32,514 PM [cluster-test-shard-dispatcher-24] [INFO] RaftActor - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 06:45:32,515 PM [cluster-test-shard-dispatcher-20] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 06:45:32,516 PM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 06:45:32,516 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:32,517 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:32,517 PM [cluster-test-shard-dispatcher-24] [INFO] PekkoRecovery - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:32,517 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:32,520 PM [cluster-test-shard-dispatcher-24] [INFO] PekkoRecovery - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 06:45:32,520 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 06:45:32,520 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards is initialized 06:45:32,520 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 06:45:32,521 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:32,521 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 06:45:32,521 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 06:45:32,521 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards from null to Follower 06:45:32,522 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 06:45:32,522 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 06:45:32,522 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards is initialized 06:45:32,523 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 06:45:32,523 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:32,523 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 06:45:32,523 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 06:45:32,523 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards from null to Follower 06:45:32,524 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 06:45:32,535 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-4517942976335587314] 06:45:32,535 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 06:45:32,536 PM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:45:32,536 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 06:45:32,540 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:45:32,540 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:45:32,549 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:32,551 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:32,555 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForAllShards 06:45:32,568 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1042369172] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:32,569 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 06:45:32,569 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 06:45:32,586 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 06:45:32,586 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:32,586 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:32,590 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 06:45:32,591 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} started 06:45:32,592 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@38fec521} 06:45:32,592 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:32,592 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 06:45:32,593 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:32,596 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForAllShards 06:45:32,596 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:32,596 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:32,598 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#414239484] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:32,598 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 06:45:32,601 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 06:45:32,601 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 06:45:32,602 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 06:45:32,602 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#582792201] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:32,602 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#1143495624 created and ready for shard:member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:32,603 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#1143495624] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:32,604 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:32,605 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 06:45:32,605 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 06:45:32,605 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 06:45:32,605 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:32,605 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} started 06:45:32,607 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#705189675 created and ready for shard:member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:32,607 PM [cluster-test-shard-dispatcher-29] [INFO] RaftActor - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 06:45:32,608 PM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 06:45:32,609 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 06:45:32,609 PM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards#1836224531] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:32,609 PM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#705189675] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:32,609 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:32,610 PM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:32,610 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:32,610 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:32,610 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 06:45:32,610 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:32,610 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:32,610 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:32,611 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:32,611 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:32,612 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:32,612 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:32,612 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:32,612 PM [cluster-test-shard-dispatcher-34] [INFO] PekkoRecovery - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:32,612 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@2fee6b18} 06:45:32,612 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:32,612 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:32,613 PM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 06:45:32,613 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 06:45:32,613 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 06:45:32,614 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:32,614 PM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:32,614 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 06:45:32,614 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards is initialized 06:45:32,614 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 06:45:32,614 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 06:45:32,614 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 06:45:32,614 PM [cluster-test-shard-dispatcher-34] [INFO] PekkoRecovery - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 06:45:32,614 PM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#1489144236] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:32,615 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards from null to Follower 06:45:32,615 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 06:45:32,616 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 06:45:32,616 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards is initialized 06:45:32,616 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 06:45:32,616 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:32,616 PM [cluster-test-shard-dispatcher-34] [INFO] PeerInfos - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:32,616 PM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 06:45:32,616 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:32,616 PM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 06:45:32,616 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards from null to Follower 06:45:32,616 PM [cluster-test-shard-dispatcher-34] [INFO] PekkoRecovery - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:32,616 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#2132323327 created and ready for shard:member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:32,616 PM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#2132323327] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:32,617 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 06:45:32,619 PM [cluster-test-shard-dispatcher-34] [INFO] PekkoRecovery - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 06:45:32,619 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 06:45:32,619 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:32,620 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:32,620 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:32,620 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#1489144236] 06:45:32,620 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:32,620 PM [cluster-test-shard-dispatcher-34] [INFO] PeerInfos - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:32,620 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 06:45:32,620 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:32,620 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:32,620 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: updated peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards#-1895832261] 06:45:32,620 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 06:45:32,620 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards is initialized 06:45:32,621 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 06:45:32,621 PM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 06:45:32,621 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards from null to Follower 06:45:32,621 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards#-1895832261] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:32,621 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#-83193819 created and ready for shard:member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:32,621 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#-83193819] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:32,621 PM [cluster-test-shard-dispatcher-34] [INFO] RaftActor - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 06:45:32,622 PM [cluster-test-shard-dispatcher-34] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 06:45:32,624 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:32,624 PM [cluster-test-shard-dispatcher-34] [INFO] PekkoRecovery - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:32,627 PM [cluster-test-shard-dispatcher-34] [INFO] PekkoRecovery - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 06:45:32,628 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 06:45:32,628 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:32,628 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 06:45:32,628 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards is initialized 06:45:32,628 PM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 06:45:32,628 PM [cluster-test-shard-dispatcher-34] [INFO] PeerInfos - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:32,628 PM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 06:45:32,628 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-8431338109428564332] 06:45:32,628 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 06:45:32,632 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards from null to Follower 06:45:32,636 PM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:45:32,636 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 06:45:32,637 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:45:32,637 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:45:32,647 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:32,648 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:32,650 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForAllShards 06:45:32,660 PM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1959738076] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:32,664 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 06:45:32,664 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 06:45:32,671 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 06:45:32,671 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:32,672 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:32,677 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 06:45:32,681 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 06:45:32,684 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} started 06:45:32,684 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 06:45:32,684 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:32,685 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:32,688 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@4e2041a} 06:45:32,688 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForAllShards 06:45:32,688 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:32,688 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:32,688 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:32,688 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:32,689 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:32,689 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:32,689 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:32,689 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:32,689 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 06:45:32,690 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 06:45:32,690 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#1797933296 created and ready for shard:member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:32,690 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 06:45:32,690 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#-692956029] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:32,691 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards#-1056722674] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:32,691 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 06:45:32,692 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 06:45:32,693 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 06:45:32,693 PM [cluster-test-shard-dispatcher-16] [INFO] RaftActor - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 06:45:32,694 PM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 06:45:32,694 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:32,695 PM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:32,696 PM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#1797933296] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:32,696 PM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-922050362] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:32,696 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#-593992662 created and ready for shard:member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:32,696 PM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#-593992662] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:32,697 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 06:45:32,697 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:32,697 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:32,698 PM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:32,698 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:32,698 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:32,698 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:32,698 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:32,698 PM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 06:45:32,699 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 06:45:32,699 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:32,699 PM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 06:45:32,699 PM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:32,699 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 06:45:32,699 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards is initialized 06:45:32,699 PM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 06:45:32,699 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards from null to Follower 06:45:32,700 PM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 06:45:32,701 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 06:45:32,701 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards is initialized 06:45:32,701 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 06:45:32,701 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:32,701 PM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 06:45:32,701 PM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:32,702 PM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 06:45:32,702 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards from null to Follower 06:45:32,702 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 06:45:32,702 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@70b671a5} 06:45:32,702 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} started 06:45:32,703 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:32,703 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:32,703 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 06:45:32,703 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:32,703 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:32,703 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-2110715025] 06:45:32,703 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:32,703 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:32,703 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:32,703 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#474608810] 06:45:32,704 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 06:45:32,704 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 06:45:32,704 PM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-2110715025] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:32,704 PM [cluster-test-shard-dispatcher-29] [INFO] RaftActor - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 06:45:32,704 PM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#474608810] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:32,704 PM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#-969138694 created and ready for shard:member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:32,704 PM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#-969138694] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:32,705 PM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#99866861 created and ready for shard:member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:32,705 PM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#99866861] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:32,705 PM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 06:45:32,705 PM [cluster-test-shard-dispatcher-30] [INFO] RaftActor - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 06:45:32,705 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:32,706 PM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:32,706 PM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 06:45:32,707 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:32,707 PM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:32,708 PM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 06:45:32,709 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 06:45:32,709 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:32,709 PM [cluster-test-shard-dispatcher-16] [INFO] PeerInfos - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:32,709 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 06:45:32,709 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards is initialized 06:45:32,709 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 06:45:32,709 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 06:45:32,709 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards from null to Follower 06:45:32,709 PM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 06:45:32,710 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 06:45:32,710 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards is initialized 06:45:32,710 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 06:45:32,710 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:32,710 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 06:45:32,710 PM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:32,710 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 06:45:32,710 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards from null to Follower 06:45:32,858 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 06:45:32,858 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:45:32,858 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 06:45:32,858 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:32,858 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 06:45:32,858 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards from Follower to Candidate 06:45:32,897 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 06:45:32,897 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:45:32,897 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 06:45:32,897 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:32,897 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 06:45:32,898 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards from Follower to Candidate 06:45:32,907 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 06:45:32,907 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:45:32,907 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 06:45:32,907 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:32,907 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 06:45:32,907 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards from Follower to Candidate 06:45:32,928 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 06:45:32,928 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:45:32,928 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 06:45:32,928 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:32,928 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 06:45:32,928 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards from Follower to Candidate 06:45:33,238 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 06:45:33,257 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 06:45:33,307 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 06:45:33,327 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 06:45:33,476 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 06:45:33,476 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 06:45:33,476 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:33,476 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:33,476 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:33,476 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:33,477 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:33,477 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:33,477 PM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:33,477 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:33,477 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:33,477 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:33,477 PM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:33,477 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:33,477 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:33,477 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:33,477 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:33,477 PM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:33,477 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:33,477 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:33,477 PM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:33,477 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:33,477 PM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:33,477 PM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:33,477 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:33,477 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:33,477 PM [cluster-test-shard-dispatcher-24] [INFO] PeerInfos - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:33,478 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:33,478 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:33,479 PM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:33,483 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:33,484 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:33,484 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:33,484 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:33,484 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:33,484 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:33,484 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:33,484 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:33,484 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:33,484 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:33,484 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:33,484 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:33,484 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:33,484 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:33,484 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:33,484 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:33,484 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:33,484 PM [cluster-test-shard-dispatcher-34] [INFO] PeerInfos - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:33,484 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:33,484 PM [cluster-test-shard-dispatcher-34] [INFO] PeerInfos - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:33,484 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:33,484 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:33,484 PM [cluster-test-shard-dispatcher-34] [INFO] PeerInfos - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:33,484 PM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:33,579 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:33,579 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:33,580 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:33,580 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:33,580 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:33,580 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:33,580 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:33,580 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:33,580 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:33,580 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:33,580 PM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:33,580 PM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:33,580 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:33,580 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:33,580 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:33,580 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:33,580 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:33,580 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:33,580 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:33,580 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:33,580 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:33,580 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:33,580 PM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:33,580 PM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:33,611 PM [cluster-test-shard-dispatcher-36] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 06:45:33,613 PM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:33,613 PM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:33,622 PM [cluster-test-shard-dispatcher-36] [INFO] RaftActorBehavior - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 06:45:33,623 PM [cluster-test-shard-dispatcher-36] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 06:45:33,623 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 06:45:33,623 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 06:45:33,623 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@39eb1192 06:45:33,623 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards from Candidate to Leader 06:45:33,624 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@15c98e3f 06:45:33,624 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done false 06:45:33,624 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done false 06:45:33,624 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5bc31cdd 06:45:33,650 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 06:45:33,658 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 06:45:33,660 PM [cluster-test-shard-dispatcher-21] [INFO] Follower - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:33,660 PM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:33,667 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 06:45:33,668 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 06:45:33,668 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 06:45:33,668 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 06:45:33,668 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@df7e693 06:45:33,668 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards from Candidate to Leader 06:45:33,669 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done false 06:45:33,669 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1faf56e3 06:45:33,669 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done false 06:45:33,669 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@24da5ed7 06:45:33,687 PM [cluster-test-shard-dispatcher-36] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 06:45:33,689 PM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:33,689 PM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:33,696 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 06:45:33,696 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 06:45:33,696 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 06:45:33,696 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 06:45:33,696 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@67c70d87 06:45:33,696 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards from Candidate to Leader 06:45:33,696 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testChangeMemberVotingStatesForAllShards is ready 06:45:33,697 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done false 06:45:33,697 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done false 06:45:33,697 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1c822414 06:45:33,697 PM [cluster-test-shard-dispatcher-36] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 06:45:33,697 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testChangeMemberVotingStatesForAllShards is ready 06:45:33,697 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2a1da43c 06:45:33,697 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testChangeMemberVotingStatesForAllShards is ready 06:45:33,699 PM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:33,699 PM [cluster-test-shard-dispatcher-21] [INFO] Follower - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:33,707 PM [cluster-test-shard-dispatcher-36] [INFO] RaftActorBehavior - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 06:45:33,707 PM [cluster-test-shard-dispatcher-36] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 06:45:33,707 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 06:45:33,707 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 06:45:33,707 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@372cf8df 06:45:33,707 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards from Candidate to Leader 06:45:33,707 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testChangeMemberVotingStatesForAllShards is ready 06:45:33,707 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 06:45:33,707 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 06:45:33,708 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 06:45:33,708 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 06:45:33,708 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done false 06:45:33,708 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done false 06:45:33,708 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2e2235a7 06:45:33,708 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5b2d4f6d 06:45:33,708 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testChangeMemberVotingStatesForAllShards is ready 06:45:33,708 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testChangeMemberVotingStatesForAllShards is ready 06:45:33,708 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 06:45:33,709 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 06:45:33,709 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 06:45:33,709 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1829486303] 06:45:33,709 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:45:33,709 PM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#-1806804083] 06:45:33,709 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#-1806804083] 06:45:33,709 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1042369172] 06:45:33,709 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 06:45:33,709 PM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#582792201] 06:45:33,709 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#582792201] 06:45:33,709 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1959738076] 06:45:33,709 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 06:45:33,710 PM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#-692956029] 06:45:33,710 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#-692956029] 06:45:33,716 PM [main] [INFO] ClusterAdminRpcService - Change member voting states for all shards 06:45:33,716 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testChangeMemberVotingStatesForAllShards 06:45:33,717 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testChangeMemberVotingStatesForAllShards 06:45:33,717 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-2=false, member-3=false}] 06:45:33,717 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-2=false, member-3=false}] 06:45:33,718 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=people, meberVotingStatusMap={member-2=false, member-3=false}] 06:45:33,718 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=people, meberVotingStatusMap={member-2=false, member-3=false}] 06:45:33,718 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 06:45:33,719 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 06:45:33,719 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 06:45:33,719 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 06:45:33,719 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards=false, member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:33,719 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:33,719 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:33,719 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:33,721 PM [cluster-test-shard-dispatcher-35] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 06:45:33,723 PM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards} from local shard pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:33,723 PM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard cars 06:45:33,723 PM [cluster-test-shard-dispatcher-35] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 06:45:33,723 PM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 06:45:33,723 PM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 06:45:33,724 PM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards} from local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:33,724 PM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 06:45:33,724 PM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 06:45:33,724 PM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 06:45:33,724 PM [cluster-test-shard-dispatcher-35] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 06:45:33,726 PM [cluster-test-shard-dispatcher-35] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 06:45:33,726 PM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards} from local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:33,726 PM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard people 06:45:33,726 PM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 06:45:33,726 PM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 06:45:33,727 PM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards} from local shard pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:33,727 PM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard people 06:45:33,727 PM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 06:45:33,727 PM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 06:45:33,729 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1829486303] 06:45:33,729 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:45:33,729 PM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#-1806804083] 06:45:33,729 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#-1806804083] 06:45:33,730 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1829486303] 06:45:33,730 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 06:45:33,730 PM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#-1270573800] 06:45:33,730 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#-1270573800] 06:45:33,730 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#1185558638] 06:45:33,730 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 06:45:33,730 PM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-1279939738] 06:45:33,730 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-1279939738] 06:45:33,730 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#1185558638] 06:45:33,731 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 06:45:33,731 PM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#-1704013988] 06:45:33,731 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#-1704013988] 06:45:33,731 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1042369172] 06:45:33,731 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 06:45:33,731 PM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#582792201] 06:45:33,731 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#582792201] 06:45:33,731 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done true 06:45:33,731 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done true 06:45:33,731 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done true 06:45:33,732 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1042369172] 06:45:33,732 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 06:45:33,732 PM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards#1836224531] 06:45:33,732 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards#1836224531] 06:45:33,734 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done true 06:45:33,734 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done true 06:45:33,734 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#414239484] 06:45:33,734 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 06:45:33,734 PM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#1489144236] 06:45:33,734 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#1489144236] 06:45:33,734 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#414239484] 06:45:33,734 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 06:45:33,734 PM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards#-1895832261] 06:45:33,735 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards#-1895832261] 06:45:33,735 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done true 06:45:33,735 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done true 06:45:33,735 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1959738076] 06:45:33,735 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 06:45:33,735 PM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#-692956029] 06:45:33,735 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#-692956029] 06:45:33,735 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1959738076] 06:45:33,735 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 06:45:33,735 PM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards#-1056722674] 06:45:33,736 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards#-1056722674] 06:45:33,736 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-922050362] 06:45:33,736 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 06:45:33,736 PM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-2110715025] 06:45:33,736 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-2110715025] 06:45:33,736 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-922050362] 06:45:33,736 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 06:45:33,736 PM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#474608810] 06:45:33,736 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#474608810] 06:45:33,737 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done true 06:45:33,737 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 06:45:33,737 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:33,738 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:33,738 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 06:45:33,738 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Stopping Shard 06:45:33,738 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Stopping Shard 06:45:33,738 PM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: journal closed 06:45:33,738 PM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: journal closed 06:45:33,738 PM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 06:45:33,738 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 06:45:33,739 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 06:45:33,739 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:33,739 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:33,739 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 06:45:33,739 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Stopping Shard 06:45:33,739 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Stopping Shard 06:45:33,739 PM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: journal closed 06:45:33,740 PM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: journal closed 06:45:33,740 PM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 06:45:33,740 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 06:45:33,741 PM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:45:33,741 PM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [08/07/2025 18:45:33.744] [cluster-test-pekko.remote.default-remote-dispatcher-6] [org.apache.pekko.stream.Log(pekko://cluster-test/system/Materializers/StreamSupervisor-70)] [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [08/07/2025 18:45:33.744] [cluster-test-pekko.remote.default-remote-dispatcher-9] [org.apache.pekko.stream.Log(pekko://cluster-test/system/Materializers/StreamSupervisor-70)] [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [08/07/2025 18:45:33.744] [cluster-test-pekko.remote.default-remote-dispatcher-6] [org.apache.pekko.stream.Log(pekko://cluster-test/system/Materializers/StreamSupervisor-70)] [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [08/07/2025 18:45:33.745] [cluster-test-pekko.remote.default-remote-dispatcher-9] [org.apache.pekko.stream.Log(pekko://cluster-test/system/Materializers/StreamSupervisor-70)] [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:33,760 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 06:45:33,761 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:33,761 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:33,761 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 06:45:33,762 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Stopping Shard 06:45:33,762 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Stopping Shard 06:45:33,762 PM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: journal closed 06:45:33,763 PM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: journal closed 06:45:33,763 PM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 06:45:33,763 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 06:45:33,764 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 06:45:33,764 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:33,764 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:33,764 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 06:45:33,765 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Stopping Shard 06:45:33,765 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Stopping Shard 06:45:33,765 PM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: journal closed 06:45:33,765 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: journal closed 06:45:33,765 PM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 06:45:33,765 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 06:45:33,766 PM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:45:33,766 PM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:45:34,495 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 06:45:34,754 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-8431338109428564332] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 06:45:34,785 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-4517942976335587314] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 06:45:34,788 PM [cluster-test-pekko.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:34,792 PM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/07/2025 18:45:34.794] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:45:34,805 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 06:45:34,805 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 06:45:34,805 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 06:45:34,805 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 06:45:34,806 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@33137ae2 06:45:34,806 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardDataTree - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@33137ae2 06:45:34,806 PM [cluster-test-shard-dispatcher-36] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 06:45:34,806 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5e3c33 06:45:34,806 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardDataTree - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5e3c33 06:45:34,806 PM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 06:45:34,806 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@399de9e6 06:45:34,806 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@105ea223 06:45:35,215 PM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: leader transfer aborted 06:45:35,215 PM [cluster-test-shard-dispatcher-35] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 409.0 ms 06:45:35,215 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: In unpauseLeader 06:45:35,215 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Stopping Shard 06:45:35,216 PM [cluster-test-shard-dispatcher-35] [INFO] EnabledRaftStorage - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: journal closed 06:45:35,304 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: leader transfer aborted 06:45:35,304 PM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 498.7 ms 06:45:35,304 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: In unpauseLeader 06:45:35,305 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Stopping Shard 06:45:35,306 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: journal closed 06:45:35,306 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 06:45:35,307 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 06:45:35,307 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 06:45:35,307 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 06:45:35,307 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 06:45:35,307 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 06:45:35,308 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7611f2e4 06:45:35,308 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@697fc69c 06:45:35,308 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@17971a35 06:45:35,308 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@697fc69c 06:45:35,308 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 06:45:35,308 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3c107a6d 06:45:35,308 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardDataTree - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3c107a6d 06:45:35,308 PM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 06:45:35,615 PM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: leader transfer aborted 06:45:35,615 PM [cluster-test-shard-dispatcher-35] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 307.3 ms 06:45:35,615 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: In unpauseLeader 06:45:35,616 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Stopping Shard 06:45:35,616 PM [cluster-test-shard-dispatcher-35] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: journal closed 06:45:35,904 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: leader transfer aborted 06:45:35,904 PM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 596.9 ms 06:45:35,905 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: In unpauseLeader 06:45:35,905 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Stopping Shard 06:45:35,906 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: journal closed 06:45:35,906 PM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 06:45:35,907 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 06:45:35,908 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:45:35,908 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:45:35,911 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/07/2025 18:45:35.913] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:45:35,940 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 06:45:35,949 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-5202383136713062473] 06:45:35,949 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 06:45:35,952 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 06:45:35,952 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 06:45:35,952 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:45:35,952 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:45:35,963 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:35,965 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 06:45:35,965 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 06:45:35,965 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:35,965 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 06:45:35,967 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testGetPrefixShardRole 06:45:35,970 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#883382874] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:35,970 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 06:45:35,971 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 06:45:35,976 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 06:45:35,976 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-default-config_testGetPrefixShardRole 06:45:35,976 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:35,976 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole 06:45:35,979 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testGetPrefixShardRole: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testGetPrefixShardRole, generation=0} 06:45:35,980 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@245f671c} 06:45:35,980 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testGetPrefixShardRole, generation=0} started 06:45:35,980 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:35,980 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config_testGetPrefixShardRole 06:45:35,980 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:35,981 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole 06:45:35,981 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole to pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole 06:45:35,981 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-default-config_testGetPrefixShardRole on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole on shard member-1-shard-default-config_testGetPrefixShardRole whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole#165020168] 06:45:35,982 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole#165020168] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:35,982 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-default-config_testGetPrefixShardRole: Shard created, persistent : true 06:45:35,982 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:35,982 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole-notifier#-640978753 created and ready for shard:member-1-shard-default-config_testGetPrefixShardRole 06:45:35,982 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole-notifier#-640978753] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:35,984 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testGetPrefixShardRole 06:45:35,986 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole#-940928840] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:35,986 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 06:45:35,986 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 06:45:35,987 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 06:45:35,987 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-default-oper_testGetPrefixShardRole 06:45:35,987 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:35,987 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-oper_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole 06:45:35,990 PM [cluster-test-shard-dispatcher-16] [INFO] RaftActor - member-1-shard-default-config_testGetPrefixShardRole: Starting recovery with journal batch size 1 06:45:35,991 PM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-default-config_testGetPrefixShardRole: journal open: applyTo=0 06:45:35,991 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testGetPrefixShardRole: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testGetPrefixShardRole, generation=0} 06:45:35,992 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@e36cc10} 06:45:35,992 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:35,993 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-default-config_testGetPrefixShardRole: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:35,993 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testGetPrefixShardRole, generation=0} started 06:45:35,995 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-default-config_testGetPrefixShardRole: Local TermInfo store seeded with TermInfo{term=0} 06:45:35,996 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:35,996 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-default-oper_testGetPrefixShardRole 06:45:35,996 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-config_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 06:45:35,996 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:35,997 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , received role change from null to Follower 06:45:35,998 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole#-489443] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:35,998 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-default-oper_testGetPrefixShardRole: Shard created, persistent : true 06:45:35,999 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole-notifier#-1620179307 created and ready for shard:member-1-shard-default-oper_testGetPrefixShardRole 06:45:35,999 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole-notifier#-1620179307] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:35,999 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActor - member-1-shard-default-oper_testGetPrefixShardRole: Starting recovery with journal batch size 1 06:45:35,999 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-default-config_testGetPrefixShardRole (Candidate): Starting new election term 1 06:45:35,999 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-default-config_testGetPrefixShardRole (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:45:35,999 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-config_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 06:45:35,999 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:36,0 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , received role change from Follower to Candidate 06:45:36,0 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-default-config_testGetPrefixShardRole (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 06:45:36,0 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-config_testGetPrefixShardRole: onLeadershipChange, isLeader: true, hasLeader : true 06:45:36,0 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: became leader with frontend state for [] 06:45:36,0 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , received role change from Candidate to Leader 06:45:36,0 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [default] 06:45:36,0 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-default-config_testGetPrefixShardRole is initialized 06:45:36,1 PM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-default-oper_testGetPrefixShardRole: journal open: applyTo=0 06:45:36,2 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , registered listener pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole 06:45:36,2 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:36,2 PM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-default-oper_testGetPrefixShardRole: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:36,5 PM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-default-oper_testGetPrefixShardRole: Local TermInfo store seeded with TermInfo{term=0} 06:45:36,6 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-oper_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 06:45:36,6 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:36,6 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , received role change from null to Follower 06:45:36,7 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-oper_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole 06:45:36,7 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-oper_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole to pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole 06:45:36,7 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-default-oper_testGetPrefixShardRole on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole on shard member-1-shard-default-oper_testGetPrefixShardRole whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole#-489443] 06:45:36,7 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [default] 06:45:36,7 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-default-oper_testGetPrefixShardRole is initialized 06:45:36,7 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6ba597dc 06:45:36,7 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-default-config_testGetPrefixShardRole from Candidate to Leader 06:45:36,7 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testGetPrefixShardRole is ready 06:45:36,9 PM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-default-oper_testGetPrefixShardRole (Candidate): Starting new election term 1 06:45:36,9 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-default-oper_testGetPrefixShardRole (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:45:36,9 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-oper_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 06:45:36,9 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:36,9 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , received role change from Follower to Candidate 06:45:36,9 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-default-oper_testGetPrefixShardRole (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 06:45:36,9 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-oper_testGetPrefixShardRole: onLeadershipChange, isLeader: true, hasLeader : true 06:45:36,9 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: became leader with frontend state for [] 06:45:36,9 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , received role change from Candidate to Leader 06:45:36,9 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , registered listener pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole 06:45:36,9 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7d96aaec 06:45:36,10 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-default-oper_testGetPrefixShardRole from Candidate to Leader 06:45:36,10 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testGetPrefixShardRole is ready 06:45:36,44 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#883382874] 06:45:36,44 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : default 06:45:36,44 PM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole#165020168] 06:45:36,45 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testGetPrefixShardRole, generation=0} 06:45:36,45 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-default-config_testGetPrefixShardRole 06:45:36,45 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@2423b45d 06:45:36,46 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-default-config_testGetPrefixShardRole: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@2423b45d 06:45:36,46 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-default-config_testGetPrefixShardRole: Stopping Shard 06:45:36,46 PM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-default-config_testGetPrefixShardRole: journal closed 06:45:36,46 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 06:45:36,47 PM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 06:45:36,47 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 06:45:36,48 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testGetPrefixShardRole, generation=0} 06:45:36,48 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda/0x00007f4ea8809e48] to Actor[pekko://cluster-test/user/$a#2073147951] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/$a#2073147951] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:36,48 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#883382874] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#883382874] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:36,48 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-default-oper_testGetPrefixShardRole 06:45:36,48 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#883382874] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#883382874] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:36,49 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@5962cfca 06:45:36,49 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-default-oper_testGetPrefixShardRole: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@5962cfca 06:45:36,49 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 1 shards 06:45:36,49 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-default-oper_testGetPrefixShardRole: Stopping Shard 06:45:36,49 PM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-default-oper_testGetPrefixShardRole: journal closed 06:45:36,50 PM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 06:45:36,50 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 06:45:36,51 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:45:36,52 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:45:36,54 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 06:45:36,69 PM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 06:45:36,83 PM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-6589692522455117636] 06:45:36,84 PM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 06:45:36,86 PM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 06:45:36,86 PM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 06:45:36,87 PM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:45:36,87 PM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:45:36,102 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:36,103 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:36,105 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,105 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 06:45:36,105 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 06:45:36,105 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 06:45:36,109 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 06:45:36,109 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 06:45:36,110 PM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#1248093591] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:36,112 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 06:45:36,112 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,113 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,114 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:36,114 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,114 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,115 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 06:45:36,115 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@14e87308} 06:45:36,116 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 06:45:36,116 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:36,118 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:36,120 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,121 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:36,121 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,122 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,122 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,122 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-1485671172] 06:45:36,122 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,122 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 06:45:36,122 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,122 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,122 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#138524177] 06:45:36,123 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 06:45:36,123 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActor - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 06:45:36,123 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 06:45:36,125 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 06:45:36,126 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 06:45:36,126 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 06:45:36,127 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1540763337 created and ready for shard:member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,128 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-2034375200 created and ready for shard:member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,128 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-1485671172] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:36,128 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#138524177] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:36,128 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#1531620869] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:36,128 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1540763337] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:36,128 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-2034375200] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:36,129 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:45:36,129 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:45:36,129 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:36,129 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:45:36,131 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 06:45:36,131 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,132 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,132 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:36,132 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,132 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,132 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 06:45:36,132 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 06:45:36,132 PM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 06:45:36,133 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 06:45:36,134 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 06:45:36,137 PM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 4.271 ms, resetting timer for the next recovery snapshot 06:45:36,137 PM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 06:45:36,138 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:45:36,138 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:45:36,139 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:36,139 PM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:45:36,139 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@1b947cac} 06:45:36,139 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:36,139 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,139 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,140 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,140 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#617871276] 06:45:36,140 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,140 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,140 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,140 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#173059520] 06:45:36,140 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 06:45:36,141 PM [cluster-test-shard-dispatcher-19] [INFO] RaftActor - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 06:45:36,141 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#617871276] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:36,141 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 06:45:36,141 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#231879691 created and ready for shard:member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,141 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#231879691] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:36,141 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 06:45:36,142 PM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 06:45:36,142 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 06:45:36,142 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#173059520] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:36,143 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1947542749 created and ready for shard:member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,143 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1947542749] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:36,143 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 06:45:36,143 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:36,143 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 06:45:36,144 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,144 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 06:45:36,144 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:45:36,144 PM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 06:45:36,144 PM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 06:45:36,144 PM [cluster-test-shard-dispatcher-18] [INFO] Recovery - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 06:45:36,145 PM [cluster-test-shard-dispatcher-20] [INFO] RaftActor - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 06:45:36,146 PM [cluster-test-shard-dispatcher-20] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 06:45:36,147 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:45:36,147 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:45:36,147 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:36,147 PM [cluster-test-shard-dispatcher-20] [INFO] PekkoRecovery - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:45:36,148 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:45:36,148 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:36,148 PM [cluster-test-shard-dispatcher-23] [INFO] PekkoRecovery - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:45:36,149 PM [cluster-test-shard-dispatcher-18] [INFO] Recovery - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 5.066 ms, resetting timer for the next recovery snapshot 06:45:36,151 PM [cluster-test-shard-dispatcher-20] [INFO] PekkoRecovery - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 06:45:36,151 PM [cluster-test-shard-dispatcher-20] [INFO] PekkoRecovery - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 06:45:36,151 PM [cluster-test-shard-dispatcher-20] [INFO] Recovery - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 06:45:36,153 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 06:45:36,153 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:36,153 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 06:45:36,153 PM [cluster-test-shard-dispatcher-23] [INFO] PekkoRecovery - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 06:45:36,153 PM [cluster-test-shard-dispatcher-23] [INFO] PekkoRecovery - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 06:45:36,153 PM [cluster-test-shard-dispatcher-23] [INFO] Recovery - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 06:45:36,153 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 06:45:36,153 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 06:45:36,154 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,154 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 06:45:36,154 PM [cluster-test-shard-dispatcher-20] [INFO] Recovery - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 2.857 ms, resetting timer for the next recovery snapshot 06:45:36,158 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 06:45:36,158 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:36,158 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 06:45:36,159 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 06:45:36,159 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 06:45:36,159 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,159 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 06:45:36,161 PM [cluster-test-shard-dispatcher-23] [INFO] Recovery - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 7.083 ms, resetting timer for the next recovery snapshot 06:45:36,162 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 06:45:36,165 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 06:45:36,165 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 06:45:36,165 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 06:45:36,165 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:36,165 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 06:45:36,165 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,166 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 06:45:36,172 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [5522924496759072815] 06:45:36,172 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 06:45:36,173 PM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:45:36,173 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 06:45:36,180 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:45:36,180 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:45:36,184 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:36,187 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:36,189 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,195 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 06:45:36,195 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 06:45:36,195 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-934420984] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:36,196 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 06:45:36,196 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,197 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,200 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 06:45:36,201 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@10c6ea17} 06:45:36,201 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 06:45:36,202 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:36,202 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:36,206 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,207 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,207 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 06:45:36,208 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 06:45:36,210 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 06:45:36,210 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:36,212 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 06:45:36,213 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,213 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1020195020 created and ready for shard:member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,213 PM [cluster-test-shard-dispatcher-14] [INFO] RaftActor - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 06:45:36,213 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#205370270] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:36,213 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#2035827431] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:36,213 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1020195020] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:36,214 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1133944634] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:36,214 PM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1133944634 created and ready for shard:member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,214 PM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 06:45:36,215 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:45:36,216 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:45:36,216 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#196835442] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:36,216 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:45:36,216 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:36,216 PM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:45:36,216 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 06:45:36,216 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 06:45:36,217 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 06:45:36,217 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,217 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,219 PM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 06:45:36,219 PM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 06:45:36,219 PM [cluster-test-shard-dispatcher-14] [INFO] Recovery - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 06:45:36,220 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 06:45:36,220 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:45:36,220 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@1bf60b7f} 06:45:36,221 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 06:45:36,221 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:36,221 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,221 PM [cluster-test-shard-dispatcher-14] [INFO] Recovery - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 2.332 ms, resetting timer for the next recovery snapshot 06:45:36,221 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,222 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1338230154] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:36,222 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 06:45:36,223 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 06:45:36,223 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1069429824 created and ready for shard:member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,223 PM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1069429824] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:36,226 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 06:45:36,226 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 06:45:36,226 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1264236971] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:36,227 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-450319697 created and ready for shard:member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,227 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 06:45:36,227 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-450319697] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:36,228 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 06:45:36,228 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 06:45:36,228 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 06:45:36,228 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:45:36,229 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:45:36,229 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:36,229 PM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:45:36,231 PM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 06:45:36,231 PM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 06:45:36,231 PM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 06:45:36,232 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:36,232 PM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:45:36,234 PM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 3.022 ms, resetting timer for the next recovery snapshot 06:45:36,235 PM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 06:45:36,235 PM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 06:45:36,235 PM [cluster-test-shard-dispatcher-30] [INFO] Recovery - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 06:45:36,237 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 06:45:36,237 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:36,237 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 06:45:36,238 PM [cluster-test-shard-dispatcher-30] [INFO] Recovery - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 3.709 ms, resetting timer for the next recovery snapshot 06:45:36,242 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 06:45:36,242 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:36,242 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 06:45:36,242 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 06:45:36,242 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 06:45:36,242 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,242 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 06:45:36,242 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 06:45:36,242 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 06:45:36,243 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,243 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 06:45:36,243 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 06:45:36,243 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:36,243 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 06:45:36,243 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,243 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 06:45:36,245 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:45:36,245 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:45:36,246 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:36,246 PM [cluster-test-shard-dispatcher-33] [INFO] PekkoRecovery - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:45:36,247 PM [cluster-test-shard-dispatcher-33] [INFO] PekkoRecovery - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 06:45:36,247 PM [cluster-test-shard-dispatcher-33] [INFO] PekkoRecovery - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 06:45:36,248 PM [cluster-test-shard-dispatcher-33] [INFO] Recovery - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 06:45:36,250 PM [cluster-test-shard-dispatcher-33] [INFO] Recovery - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 2.741 ms, resetting timer for the next recovery snapshot 06:45:36,254 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 06:45:36,254 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:36,254 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 06:45:36,254 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 06:45:36,255 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 06:45:36,255 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,255 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 06:45:36,256 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 06:45:36,262 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 06:45:36,264 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 06:45:36,265 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:36,265 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,265 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,265 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,265 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,265 PM [cluster-test-shard-dispatcher-33] [INFO] PeerInfos - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,265 PM [cluster-test-shard-dispatcher-34] [INFO] PeerInfos - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,265 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:36,265 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,265 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,265 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,265 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,265 PM [cluster-test-shard-dispatcher-14] [INFO] PeerInfos - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,266 PM [cluster-test-shard-dispatcher-32] [INFO] PeerInfos - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,273 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-7197667709586271645] 06:45:36,273 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 06:45:36,275 PM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:45:36,275 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 06:45:36,279 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:45:36,279 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:45:36,288 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:36,290 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:36,298 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,303 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-728568626] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:36,304 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 06:45:36,304 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 06:45:36,307 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 06:45:36,307 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,307 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,309 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 06:45:36,312 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@23217c2} 06:45:36,312 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 06:45:36,312 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:36,312 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,312 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:36,314 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,314 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 06:45:36,315 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:36,315 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActor - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 06:45:36,316 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#233828711] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:36,316 PM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 06:45:36,316 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 06:45:36,317 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActor - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 06:45:36,318 PM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 06:45:36,319 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#667601296 created and ready for shard:member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,319 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1238595790 created and ready for shard:member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,319 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#667601296] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:36,319 PM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#517963266] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:36,320 PM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1238595790] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:36,324 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,324 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:45:36,325 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:45:36,331 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1226674354] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:36,331 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:45:36,331 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 06:45:36,332 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 06:45:36,333 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:36,333 PM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:45:36,334 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 06:45:36,334 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,334 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,334 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:45:36,335 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 06:45:36,336 PM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 06:45:36,336 PM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 06:45:36,336 PM [cluster-test-shard-dispatcher-18] [INFO] Recovery - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 06:45:36,337 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 06:45:36,337 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@79d528f7} 06:45:36,338 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 06:45:36,339 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:36,339 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:36,339 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,339 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,339 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,339 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,339 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,339 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,339 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:36,339 PM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:45:36,339 PM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 06:45:36,339 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:36,339 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,340 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,340 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,340 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader on shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1589672521] 06:45:36,340 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,341 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,341 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,341 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader on shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-49330264] 06:45:36,341 PM [cluster-test-shard-dispatcher-18] [INFO] Recovery - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 5.124 ms, resetting timer for the next recovery snapshot 06:45:36,341 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 06:45:36,342 PM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 06:45:36,342 PM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 06:45:36,342 PM [cluster-test-shard-dispatcher-16] [INFO] Recovery - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 06:45:36,342 PM [cluster-test-shard-dispatcher-27] [INFO] RaftActor - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 06:45:36,342 PM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#2103699778 created and ready for shard:member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,342 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 06:45:36,342 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-49330264] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:36,342 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#2103699778] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:36,342 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1589672521] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:36,343 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1897968778 created and ready for shard:member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,343 PM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 06:45:36,343 PM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1897968778] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:36,343 PM [cluster-test-shard-dispatcher-32] [INFO] RaftActor - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 06:45:36,344 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:45:36,345 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:45:36,345 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:36,345 PM [cluster-test-shard-dispatcher-33] [INFO] PekkoRecovery - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:45:36,345 PM [cluster-test-shard-dispatcher-32] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 06:45:36,345 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 06:45:36,345 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 06:45:36,345 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 06:45:36,345 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:36,346 PM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 06:45:36,346 PM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,346 PM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,346 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 06:45:36,346 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:45:36,347 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:45:36,347 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:36,347 PM [cluster-test-shard-dispatcher-32] [INFO] PekkoRecovery - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:45:36,347 PM [cluster-test-shard-dispatcher-16] [INFO] Recovery - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 5.377 ms, resetting timer for the next recovery snapshot 06:45:36,351 PM [cluster-test-shard-dispatcher-32] [INFO] PekkoRecovery - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 06:45:36,351 PM [cluster-test-shard-dispatcher-32] [INFO] PekkoRecovery - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 06:45:36,351 PM [cluster-test-shard-dispatcher-32] [INFO] Recovery - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 06:45:36,351 PM [cluster-test-shard-dispatcher-33] [INFO] PekkoRecovery - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 06:45:36,351 PM [cluster-test-shard-dispatcher-33] [INFO] PekkoRecovery - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 06:45:36,351 PM [cluster-test-shard-dispatcher-33] [INFO] Recovery - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 06:45:36,352 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 06:45:36,352 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 06:45:36,352 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 06:45:36,352 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:36,352 PM [cluster-test-shard-dispatcher-16] [INFO] PeerInfos - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,352 PM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 06:45:36,352 PM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,352 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 06:45:36,353 PM [cluster-test-shard-dispatcher-32] [INFO] Recovery - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 2.573 ms, resetting timer for the next recovery snapshot 06:45:36,356 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 06:45:36,356 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 06:45:36,356 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 06:45:36,356 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:36,356 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 06:45:36,357 PM [cluster-test-shard-dispatcher-33] [INFO] Recovery - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 5.613 ms, resetting timer for the next recovery snapshot 06:45:36,357 PM [cluster-test-shard-dispatcher-32] [INFO] PeerInfos - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,357 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,357 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 06:45:36,360 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 06:45:36,360 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 06:45:36,360 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 06:45:36,360 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:36,360 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 06:45:36,360 PM [cluster-test-shard-dispatcher-33] [INFO] PeerInfos - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,360 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:36,360 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 06:45:37,98 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 06:45:37,98 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 06:45:37,99 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:37,99 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,99 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,99 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,99 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,99 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:37,99 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:37,99 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,99 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,99 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,99 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,99 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,99 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,99 PM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,99 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,99 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,99 PM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,99 PM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,99 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:37,99 PM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,99 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,99 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,99 PM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,99 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,99 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,99 PM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,99 PM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,99 PM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,107 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:37,107 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,107 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,107 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:37,107 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,107 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,107 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,107 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,107 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,107 PM [cluster-test-shard-dispatcher-14] [INFO] PeerInfos - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,107 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,107 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:37,107 PM [cluster-test-shard-dispatcher-14] [INFO] PeerInfos - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,107 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,107 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:37,107 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,107 PM [cluster-test-shard-dispatcher-14] [INFO] PeerInfos - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,107 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,107 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,107 PM [cluster-test-shard-dispatcher-14] [INFO] PeerInfos - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,107 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,107 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,107 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,107 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,141 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#1248093591] 06:45:37,141 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:45:37,141 PM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-1485671172] 06:45:37,141 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-1485671172] 06:45:37,142 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#1248093591] 06:45:37,142 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:45:37,143 PM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-1485671172] 06:45:37,143 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-1485671172] 06:45:37,143 PM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 06:45:37,143 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,143 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,143 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 06:45:37,143 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 06:45:37,144 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 06:45:37,144 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 06:45:37,144 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 06:45:37,144 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 06:45:37,144 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:45:37,145 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 06:45:37,145 PM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,146 PM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,147 PM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,148 PM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,150 PM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 06:45:37,151 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 06:45:37,151 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 06:45:37,151 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:37,151 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 06:45:37,151 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 06:45:37,151 PM [cluster-test-shard-dispatcher-22] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 06:45:37,151 PM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 06:45:37,152 PM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 06:45:37,152 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 06:45:37,152 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:37,152 PM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 06:45:37,152 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 06:45:37,152 PM [cluster-test-shard-dispatcher-16] [INFO] Follower - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 06:45:37,152 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 06:45:37,152 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 06:45:37,152 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:37,153 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 06:45:37,153 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 06:45:37,153 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 06:45:37,153 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 06:45:37,154 PM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 06:45:37,154 PM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 06:45:37,154 PM [cluster-test-shard-dispatcher-33] [INFO] Follower - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 06:45:37,155 PM [cluster-test-shard-dispatcher-22] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 06:45:37,155 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 06:45:37,155 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:37,155 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 06:45:37,155 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 06:45:37,155 PM [cluster-test-shard-dispatcher-32] [INFO] Follower - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 06:45:37,155 PM [cluster-test-shard-dispatcher-31] [INFO] Follower - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 06:45:37,156 PM [cluster-test-shard-dispatcher-18] [INFO] Follower - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 06:45:37,164 PM [cluster-test-shard-dispatcher-22] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 06:45:37,164 PM [cluster-test-shard-dispatcher-22] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 06:45:37,167 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 06:45:37,167 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 06:45:37,167 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4d2d4118 06:45:37,167 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3fbc8593 06:45:37,167 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@127a3a24 06:45:37,167 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 06:45:37,167 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 06:45:37,166 PM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 06:45:37,167 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 06:45:37,167 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 06:45:37,167 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 06:45:37,167 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2d70c13 06:45:37,168 PM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,168 PM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard cars 06:45:37,168 PM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 06:45:37,168 PM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 06:45:37,168 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 06:45:37,166 PM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 06:45:37,169 PM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 06:45:37,169 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 06:45:37,169 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 06:45:37,169 PM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,169 PM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard people 06:45:37,169 PM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 06:45:37,169 PM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 06:45:37,169 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 06:45:37,169 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 06:45:37,169 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@a768800 06:45:37,169 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 06:45:37,170 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@eda2e58 06:45:37,170 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 06:45:37,170 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 06:45:37,170 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 06:45:37,170 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 06:45:37,170 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3bd613e3 06:45:37,170 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 06:45:37,170 PM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,170 PM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard people 06:45:37,170 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 06:45:37,170 PM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 06:45:37,170 PM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 06:45:37,170 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 06:45:37,166 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 06:45:37,171 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 06:45:37,171 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 06:45:37,171 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 06:45:37,171 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 06:45:37,171 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 06:45:37,171 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@388d62ba 06:45:37,171 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 06:45:37,171 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 06:45:37,172 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@fd77eae 06:45:37,172 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7c912914 06:45:37,172 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 06:45:37,172 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 06:45:37,173 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 06:45:37,173 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@581c625e 06:45:37,173 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 06:45:37,174 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1486df3b 06:45:37,174 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 06:45:37,174 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 06:45:37,174 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 06:45:37,174 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: VotingConfig{serverInfo=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]} has been successfully replicated to a majority of followers 06:45:37,180 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 06:45:37,180 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 06:45:37,180 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 06:45:37,179 PM [cluster-test-shard-dispatcher-20] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: VotingConfig{serverInfo=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]} has been successfully replicated to a majority of followers 06:45:37,180 PM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 06:45:37,180 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 06:45:37,180 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 06:45:37,178 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: VotingConfig{serverInfo=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]} has been successfully replicated to a majority of followers 06:45:37,177 PM [cluster-test-shard-dispatcher-22] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]} has been successfully replicated to a majority of followers 06:45:37,181 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 06:45:37,181 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 06:45:37,181 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 06:45:37,181 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 06:45:37,176 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 06:45:37,176 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 06:45:37,175 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 06:45:37,181 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 06:45:37,175 PM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,181 PM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard cars 06:45:37,181 PM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 06:45:37,181 PM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 06:45:37,181 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#1248093591] 06:45:37,181 PM [cluster-test-shard-dispatcher-22] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 06:45:37,181 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 06:45:37,181 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:45:37,181 PM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-1485671172] 06:45:37,182 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 06:45:37,182 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 06:45:37,182 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 06:45:37,182 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-1485671172] 06:45:37,182 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#1248093591] 06:45:37,182 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 06:45:37,182 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 06:45:37,183 PM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#138524177] 06:45:37,183 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 06:45:37,183 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 06:45:37,183 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#138524177] 06:45:37,183 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#1531620869] 06:45:37,183 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 06:45:37,183 PM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#617871276] 06:45:37,183 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#617871276] 06:45:37,184 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#1531620869] 06:45:37,184 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 06:45:37,184 PM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#173059520] 06:45:37,184 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#173059520] 06:45:37,184 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-934420984] 06:45:37,184 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 06:45:37,184 PM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#205370270] 06:45:37,184 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#205370270] 06:45:37,184 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-934420984] 06:45:37,184 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 06:45:37,184 PM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#2035827431] 06:45:37,184 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#2035827431] 06:45:37,185 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#196835442] 06:45:37,185 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 06:45:37,185 PM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1338230154] 06:45:37,185 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1338230154] 06:45:37,185 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#196835442] 06:45:37,185 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 06:45:37,185 PM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1264236971] 06:45:37,185 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1264236971] 06:45:37,185 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-728568626] 06:45:37,185 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 06:45:37,185 PM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#233828711] 06:45:37,185 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#233828711] 06:45:37,185 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-728568626] 06:45:37,185 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 06:45:37,185 PM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#517963266] 06:45:37,185 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#517963266] 06:45:37,185 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1226674354] 06:45:37,185 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 06:45:37,186 PM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1589672521] 06:45:37,186 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1589672521] 06:45:37,186 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1226674354] 06:45:37,186 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 06:45:37,186 PM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-49330264] 06:45:37,186 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-49330264] 06:45:37,186 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#1248093591] 06:45:37,186 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:45:37,186 PM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-1485671172] 06:45:37,186 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-1485671172] 06:45:37,187 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#1531620869] 06:45:37,187 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 06:45:37,187 PM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#617871276] 06:45:37,187 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#617871276] 06:45:37,188 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 06:45:37,189 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,190 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,190 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 06:45:37,190 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 06:45:37,190 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 06:45:37,190 PM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 06:45:37,191 PM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 06:45:37,191 PM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 06:45:37,191 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 06:45:37,191 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 06:45:37,192 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,192 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,192 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 06:45:37,192 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 06:45:37,193 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 06:45:37,193 PM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 06:45:37,193 PM [cluster-test-shard-dispatcher-33] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 06:45:37,193 PM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 06:45:37,193 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 06:45:37,196 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:45:37,196 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:45:37,203 PM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:37,203 PM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:37,203 PM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:37,203 PM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:37,204 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 06:45:37,217 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 06:45:37,218 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,218 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,218 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 06:45:37,219 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 06:45:37,219 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 06:45:37,219 PM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 06:45:37,220 PM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 06:45:37,220 PM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 06:45:37,220 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 06:45:37,220 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 06:45:37,220 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,220 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:37,220 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 06:45:37,221 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 06:45:37,221 PM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 06:45:37,221 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 06:45:37,222 PM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 06:45:37,223 PM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 06:45:37,223 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 06:45:37,224 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:45:37,224 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:45:38,117 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 06:45:38,216 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-7197667709586271645] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 06:45:38,236 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [5522924496759072815] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 06:45:38,239 PM [cluster-test-pekko.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:38,239 PM [cluster-test-pekko.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:38,241 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/07/2025 18:45:38.243] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:45:38,258 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 06:45:38,258 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:38,259 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 06:45:38,259 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 06:45:38,259 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@47547e7a 06:45:38,259 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2a8a5fb6 06:45:38,259 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@300f2c2a 06:45:38,259 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@70eca519 06:45:38,260 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@300f2c2a 06:45:38,260 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@70eca519 06:45:38,260 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 06:45:38,260 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 06:45:38,777 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 06:45:38,777 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 06:45:38,777 PM [cluster-test-shard-dispatcher-22] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 518.0 ms 06:45:38,777 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 06:45:38,777 PM [cluster-test-shard-dispatcher-23] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 517.8 ms 06:45:38,777 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 06:45:38,778 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 06:45:38,778 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 06:45:38,778 PM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 06:45:38,778 PM [cluster-test-shard-dispatcher-22] [INFO] EnabledRaftStorage - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 06:45:38,779 PM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 06:45:38,779 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 06:45:38,779 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 06:45:38,780 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:38,780 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 06:45:38,780 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 06:45:38,780 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3fef9fea 06:45:38,780 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3fef9fea 06:45:38,781 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 06:45:38,781 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2afbf75d 06:45:38,781 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@69fcbc4c 06:45:38,781 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@69fcbc4c 06:45:38,781 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@243c1c72 06:45:38,781 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 06:45:39,96 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 06:45:39,96 PM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 06:45:39,96 PM [cluster-test-shard-dispatcher-23] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 316.0 ms 06:45:39,97 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 06:45:39,96 PM [cluster-test-shard-dispatcher-34] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 316.3 ms 06:45:39,97 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 06:45:39,97 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 06:45:39,97 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 06:45:39,98 PM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 06:45:39,98 PM [cluster-test-shard-dispatcher-34] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 06:45:39,99 PM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 06:45:39,99 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 06:45:39,100 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:45:39,100 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:45:39,104 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/07/2025 18:45:39.106] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:45:39,120 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 06:45:39,127 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-6636491781373619630] 06:45:39,127 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 06:45:39,129 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 06:45:39,129 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 06:45:39,130 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:45:39,130 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:45:39,139 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:39,140 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:39,140 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 06:45:39,140 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 06:45:39,141 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 06:45:39,142 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,145 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-98551951] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:39,146 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 06:45:39,146 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 06:45:39,149 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 06:45:39,149 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,150 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,150 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:39,150 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,150 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,152 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 06:45:39,153 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 06:45:39,153 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:39,154 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:39,156 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,153 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@265bd571} 06:45:39,157 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:39,157 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,158 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,158 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,158 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#491746921] 06:45:39,158 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,158 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,158 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,158 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-1489846017] 06:45:39,158 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 06:45:39,159 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 06:45:39,159 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-192591737 created and ready for shard:member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,159 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 06:45:39,159 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#1892699574] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:39,161 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 06:45:39,161 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#491746921] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:39,161 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-1489846017] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:39,161 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-192591737] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:39,161 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1424587961 created and ready for shard:member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,162 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1424587961] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:39,161 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 06:45:39,162 PM [cluster-test-shard-dispatcher-15] [INFO] RaftActor - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 06:45:39,162 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 06:45:39,162 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 06:45:39,162 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:45:39,162 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:45:39,163 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:39,163 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:45:39,163 PM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 06:45:39,163 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 06:45:39,164 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 06:45:39,164 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,164 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,164 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:39,164 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,164 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,164 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@e2a6a5} 06:45:39,164 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:39,164 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,165 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:45:39,165 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,165 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,165 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1438027026] 06:45:39,165 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,165 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,165 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,165 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#1720862234] 06:45:39,165 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 06:45:39,166 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:45:39,166 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 06:45:39,166 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 06:45:39,166 PM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 06:45:39,166 PM [cluster-test-shard-dispatcher-14] [INFO] RaftActor - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 06:45:39,166 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1438027026] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:39,167 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1349538834 created and ready for shard:member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,167 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1349538834] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:39,167 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#1720862234] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:39,167 PM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 06:45:39,168 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 06:45:39,168 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:45:39,168 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:45:39,168 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:39,169 PM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:45:39,168 PM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 2.525 ms, resetting timer for the next recovery snapshot 06:45:39,169 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#489428735 created and ready for shard:member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,169 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#489428735] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:39,169 PM [cluster-test-shard-dispatcher-19] [INFO] RaftActor - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 06:45:39,170 PM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 06:45:39,171 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:39,171 PM [cluster-test-shard-dispatcher-15] [INFO] PekkoRecovery - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:45:39,171 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:45:39,171 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:45:39,172 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:39,172 PM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:45:39,172 PM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 06:45:39,172 PM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 06:45:39,172 PM [cluster-test-shard-dispatcher-14] [INFO] Recovery - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 06:45:39,173 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 06:45:39,173 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:39,173 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 06:45:39,174 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 06:45:39,174 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 06:45:39,174 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,174 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 06:45:39,174 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 06:45:39,174 PM [cluster-test-shard-dispatcher-15] [INFO] PekkoRecovery - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 06:45:39,174 PM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 06:45:39,174 PM [cluster-test-shard-dispatcher-15] [INFO] PekkoRecovery - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 06:45:39,174 PM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 06:45:39,174 PM [cluster-test-shard-dispatcher-15] [INFO] Recovery - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 06:45:39,174 PM [cluster-test-shard-dispatcher-19] [INFO] Recovery - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 06:45:39,175 PM [cluster-test-shard-dispatcher-14] [INFO] Recovery - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 2.867 ms, resetting timer for the next recovery snapshot 06:45:39,178 PM [cluster-test-shard-dispatcher-19] [INFO] Recovery - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 3.676 ms, resetting timer for the next recovery snapshot 06:45:39,178 PM [cluster-test-shard-dispatcher-15] [INFO] Recovery - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 3.780 ms, resetting timer for the next recovery snapshot 06:45:39,179 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 06:45:39,179 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:39,179 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 06:45:39,179 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 06:45:39,179 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 06:45:39,179 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,179 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 06:45:39,182 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 06:45:39,182 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:39,182 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 06:45:39,182 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 06:45:39,182 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-9129418018723133592] 06:45:39,182 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 06:45:39,182 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 06:45:39,182 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,182 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 06:45:39,182 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 06:45:39,182 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 06:45:39,182 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 06:45:39,182 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:39,183 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 06:45:39,183 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,183 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 06:45:39,186 PM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:45:39,186 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 06:45:39,186 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:45:39,186 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:45:39,200 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:39,202 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:39,204 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,210 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#74124179] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:39,227 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 06:45:39,228 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 06:45:39,228 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:39,230 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 06:45:39,230 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 06:45:39,230 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:39,232 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 06:45:39,232 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,233 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,233 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@7efe44aa} 06:45:39,233 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:39,233 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,233 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,234 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 06:45:39,234 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1762011251] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:39,235 PM [cluster-test-shard-dispatcher-24] [INFO] RaftActor - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 06:45:39,235 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-27658563 created and ready for shard:member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,234 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,235 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-27658563] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:39,236 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#1582585787] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:39,237 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 06:45:39,237 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 06:45:39,237 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 06:45:39,237 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#1255208011] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:39,237 PM [cluster-test-shard-dispatcher-19] [INFO] RaftActor - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 06:45:39,237 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#972079043 created and ready for shard:member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,237 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#972079043] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:39,238 PM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 06:45:39,238 PM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 06:45:39,239 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:45:39,239 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:45:39,239 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:39,239 PM [cluster-test-shard-dispatcher-24] [INFO] PekkoRecovery - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:45:39,240 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:45:39,240 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:45:39,240 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:39,240 PM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:45:39,243 PM [cluster-test-shard-dispatcher-24] [INFO] PekkoRecovery - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 06:45:39,243 PM [cluster-test-shard-dispatcher-24] [INFO] PekkoRecovery - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 06:45:39,243 PM [cluster-test-shard-dispatcher-24] [INFO] Recovery - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 06:45:39,244 PM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 06:45:39,244 PM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 06:45:39,244 PM [cluster-test-shard-dispatcher-19] [INFO] Recovery - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 06:45:39,245 PM [cluster-test-shard-dispatcher-24] [INFO] Recovery - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 2.434 ms, resetting timer for the next recovery snapshot 06:45:39,247 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 06:45:39,247 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,248 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,248 PM [cluster-test-shard-dispatcher-19] [INFO] Recovery - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 3.847 ms, resetting timer for the next recovery snapshot 06:45:39,250 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 06:45:39,250 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:39,250 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 06:45:39,250 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 06:45:39,250 PM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 06:45:39,250 PM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,250 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 06:45:39,251 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 06:45:39,251 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@4d128769} 06:45:39,252 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 06:45:39,252 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:39,252 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:39,252 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,252 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,253 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 06:45:39,253 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 06:45:39,253 PM [cluster-test-shard-dispatcher-19] [INFO] RaftActor - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 06:45:39,253 PM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 06:45:39,254 PM [cluster-test-shard-dispatcher-29] [INFO] RaftActor - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 06:45:39,254 PM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 06:45:39,255 PM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 06:45:39,255 PM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#1707897977] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:39,255 PM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-10053512] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:39,255 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-41140060 created and ready for shard:member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,255 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#74876702] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:39,255 PM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-41140060] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:39,258 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 06:45:39,260 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 06:45:39,261 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 06:45:39,261 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 06:45:39,261 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,261 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 06:45:39,262 PM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 06:45:39,264 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:39,264 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:39,264 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,264 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,264 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,264 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,264 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,264 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,264 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,264 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,264 PM [cluster-test-shard-dispatcher-24] [INFO] PeerInfos - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,264 PM [cluster-test-shard-dispatcher-24] [INFO] PeerInfos - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,266 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:45:39,266 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:45:39,266 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:39,266 PM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:45:39,266 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:45:39,266 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:45:39,267 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:39,267 PM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:45:39,270 PM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 06:45:39,270 PM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 06:45:39,270 PM [cluster-test-shard-dispatcher-29] [INFO] Recovery - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 06:45:39,270 PM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 06:45:39,270 PM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 06:45:39,270 PM [cluster-test-shard-dispatcher-30] [INFO] Recovery - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 06:45:39,271 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#74876702 created and ready for shard:member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,273 PM [cluster-test-shard-dispatcher-29] [INFO] Recovery - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 2.928 ms, resetting timer for the next recovery snapshot 06:45:39,275 PM [cluster-test-shard-dispatcher-30] [INFO] Recovery - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 4.191 ms, resetting timer for the next recovery snapshot 06:45:39,278 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 06:45:39,278 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 06:45:39,278 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 06:45:39,278 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:39,278 PM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,278 PM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 06:45:39,278 PM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,278 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 06:45:39,279 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 06:45:39,281 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 06:45:39,281 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 06:45:39,281 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 06:45:39,281 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:39,281 PM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,281 PM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 06:45:39,281 PM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,281 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 06:45:39,288 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [2850266606539582722] 06:45:39,288 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 06:45:39,290 PM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:45:39,290 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 06:45:39,290 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:45:39,295 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:45:39,303 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:39,305 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:39,307 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,312 PM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-2041762846] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:39,314 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 06:45:39,314 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 06:45:39,316 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 06:45:39,316 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,316 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,319 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 06:45:39,320 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@44c1dd08} 06:45:39,320 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 06:45:39,321 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:39,321 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,321 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:39,328 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,329 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 06:45:39,329 PM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1284935122] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:39,329 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#856567882 created and ready for shard:member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,329 PM [cluster-test-shard-dispatcher-25] [INFO] RaftActor - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 06:45:39,329 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#856567882] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:39,330 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:39,332 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,334 PM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 06:45:39,335 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 06:45:39,335 PM [cluster-test-shard-dispatcher-25] [INFO] RaftActor - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 06:45:39,336 PM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-591932693] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:39,336 PM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 06:45:39,337 PM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1952346284] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:39,338 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 06:45:39,338 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 06:45:39,338 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:45:39,338 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 06:45:39,340 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:45:39,341 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:45:39,341 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1445062750 created and ready for shard:member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,341 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:39,341 PM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1445062750] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:39,341 PM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:45:39,342 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 06:45:39,342 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,342 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,342 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:45:39,342 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:39,343 PM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:45:39,343 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 06:45:39,344 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:39,344 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,344 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,344 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:39,344 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,344 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,344 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,344 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,346 PM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 06:45:39,346 PM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 06:45:39,346 PM [cluster-test-shard-dispatcher-14] [INFO] Recovery - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 06:45:39,348 PM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 06:45:39,348 PM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 06:45:39,348 PM [cluster-test-shard-dispatcher-27] [INFO] Recovery - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 06:45:39,349 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 06:45:39,349 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@5843ed4} 06:45:39,349 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 06:45:39,350 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 06:45:39,350 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:39,350 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,350 PM [cluster-test-shard-dispatcher-14] [INFO] Recovery - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 3.959 ms, resetting timer for the next recovery snapshot 06:45:39,350 PM [cluster-test-shard-dispatcher-27] [INFO] Recovery - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 2.362 ms, resetting timer for the next recovery snapshot 06:45:39,350 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,350 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,350 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-18726096] 06:45:39,350 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,351 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 06:45:39,351 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-18726096] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:39,351 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-870006138] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:39,352 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-539775508] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:39,351 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,352 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,352 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-870006138] 06:45:39,351 PM [cluster-test-shard-dispatcher-26] [INFO] RaftActor - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 06:45:39,351 PM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-539775508 created and ready for shard:member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,351 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 06:45:39,352 PM [cluster-test-shard-dispatcher-28] [INFO] RaftActor - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 06:45:39,353 PM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 06:45:39,353 PM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1017768745 created and ready for shard:member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,353 PM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1017768745] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:39,354 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:45:39,354 PM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 06:45:39,354 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:45:39,354 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:39,354 PM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:45:39,355 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 06:45:39,355 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 06:45:39,355 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:39,355 PM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 06:45:39,356 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 06:45:39,356 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 06:45:39,356 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 06:45:39,356 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:39,356 PM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 06:45:39,356 PM [cluster-test-shard-dispatcher-14] [INFO] PeerInfos - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,356 PM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,356 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 06:45:39,358 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 06:45:39,358 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 06:45:39,358 PM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 06:45:39,358 PM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 06:45:39,358 PM [cluster-test-shard-dispatcher-26] [INFO] Recovery - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 06:45:39,359 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 06:45:39,359 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:39,359 PM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 06:45:39,359 PM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,359 PM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 06:45:39,359 PM [cluster-test-shard-dispatcher-28] [INFO] Recovery - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 06:45:39,360 PM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 06:45:39,360 PM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,360 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 06:45:39,361 PM [cluster-test-shard-dispatcher-26] [INFO] Recovery - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 2.892 ms, resetting timer for the next recovery snapshot 06:45:39,362 PM [cluster-test-shard-dispatcher-28] [INFO] Recovery - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 2.919 ms, resetting timer for the next recovery snapshot 06:45:39,365 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 06:45:39,365 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 06:45:39,365 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 06:45:39,365 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:39,366 PM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 06:45:39,366 PM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,366 PM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,366 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 06:45:39,366 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 06:45:39,366 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 06:45:39,366 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 06:45:39,366 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:39,366 PM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 06:45:39,366 PM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,366 PM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:39,366 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 06:45:39,521 PM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 06:45:39,522 PM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 06:45:39,522 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 06:45:39,522 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:39,522 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 06:45:39,522 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 06:45:39,531 PM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 06:45:39,531 PM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 06:45:39,531 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 06:45:39,531 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:39,531 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 06:45:39,531 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 06:45:39,602 PM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 06:45:39,602 PM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 06:45:39,602 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 06:45:39,602 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:39,602 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 06:45:39,602 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 06:45:39,603 PM [cluster-test-shard-dispatcher-21] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 06:45:39,603 PM [cluster-test-shard-dispatcher-21] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 06:45:39,603 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 06:45:39,603 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:39,603 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 06:45:39,604 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 06:45:39,912 PM [cluster-test-shard-dispatcher-21] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 06:45:39,931 PM [cluster-test-shard-dispatcher-21] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 06:45:39,970 PM [cluster-test-shard-dispatcher-21] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 06:45:40,1 PM [cluster-test-shard-dispatcher-21] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 06:45:40,149 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 06:45:40,150 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 06:45:40,151 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:40,151 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,151 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,151 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:40,151 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,151 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,151 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,151 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,152 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:40,152 PM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,152 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,152 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,152 PM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,152 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,152 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,152 PM [cluster-test-shard-dispatcher-14] [INFO] PeerInfos - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,152 PM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,152 PM [cluster-test-shard-dispatcher-14] [INFO] PeerInfos - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,152 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,152 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,152 PM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,152 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:40,153 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,153 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,153 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,153 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,153 PM [cluster-test-shard-dispatcher-21] [INFO] PeerInfos - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,153 PM [cluster-test-shard-dispatcher-21] [INFO] PeerInfos - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,157 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:40,157 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,157 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,157 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,157 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,157 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:40,157 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,157 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,157 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,157 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,157 PM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,157 PM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,157 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:40,157 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,157 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,157 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,157 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,157 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:40,157 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,157 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,157 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,157 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,157 PM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,157 PM [cluster-test-shard-dispatcher-19] [INFO] PeerInfos - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,212 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:40,212 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,212 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,212 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,212 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,212 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:40,212 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,212 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,212 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,212 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,212 PM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,212 PM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,212 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:40,212 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,212 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,212 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,212 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,212 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:40,212 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,212 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,212 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,212 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,212 PM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,212 PM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,271 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 06:45:40,273 PM [cluster-test-shard-dispatcher-31] [INFO] Follower - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 06:45:40,273 PM [cluster-test-shard-dispatcher-25] [INFO] Follower - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 06:45:40,281 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 06:45:40,281 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 06:45:40,281 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 06:45:40,281 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 06:45:40,282 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 06:45:40,282 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@43277faa 06:45:40,282 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 06:45:40,283 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 06:45:40,283 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 06:45:40,284 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7d614d56 06:45:40,284 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 06:45:40,284 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 06:45:40,286 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6916c528 06:45:40,286 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 06:45:40,286 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 06:45:40,286 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 06:45:40,286 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 06:45:40,310 PM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 06:45:40,352 PM [cluster-test-shard-dispatcher-21] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 06:45:40,353 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 06:45:40,354 PM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 06:45:40,355 PM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 06:45:40,355 PM [cluster-test-shard-dispatcher-31] [INFO] Follower - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 06:45:40,355 PM [cluster-test-shard-dispatcher-25] [INFO] Follower - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 06:45:40,362 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 06:45:40,363 PM [cluster-test-shard-dispatcher-21] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 06:45:40,363 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 06:45:40,363 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 06:45:40,363 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 06:45:40,363 PM [cluster-test-shard-dispatcher-21] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 06:45:40,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 06:45:40,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 06:45:40,363 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 06:45:40,363 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 06:45:40,363 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@17893f9b 06:45:40,363 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 06:45:40,363 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@143a9d32 06:45:40,363 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 06:45:40,365 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 06:45:40,365 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 06:45:40,365 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 06:45:40,365 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 06:45:40,365 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@b43c014 06:45:40,365 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@f7c55a4 06:45:40,365 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStatesWithVotingMembersDown is ready 06:45:40,365 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@73a6bc8a 06:45:40,365 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@77992626 06:45:40,365 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStatesWithVotingMembersDown is ready 06:45:40,365 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 06:45:40,365 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 06:45:40,365 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 06:45:40,365 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 06:45:40,368 PM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 06:45:40,368 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 06:45:40,368 PM [cluster-test-shard-dispatcher-21] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 06:45:40,368 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 06:45:40,368 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 06:45:40,368 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 06:45:40,368 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 06:45:40,368 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 06:45:40,368 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStatesWithVotingMembersDown is ready 06:45:40,381 PM [cluster-test-shard-dispatcher-21] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 06:45:40,382 PM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 06:45:40,382 PM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 06:45:40,387 PM [cluster-test-shard-dispatcher-21] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 06:45:40,388 PM [cluster-test-shard-dispatcher-21] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 06:45:40,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 06:45:40,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 06:45:40,388 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@581c54da 06:45:40,388 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 06:45:40,388 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 06:45:40,390 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 06:45:40,390 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 06:45:40,390 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@39d178ed 06:45:40,390 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStatesWithVotingMembersDown is ready 06:45:40,390 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 06:45:40,390 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 06:45:40,390 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6e283f2a 06:45:40,390 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStatesWithVotingMembersDown is ready 06:45:40,393 PM [cluster-test-shard-dispatcher-21] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 06:45:40,393 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 06:45:40,393 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 06:45:40,393 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 06:45:40,393 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStatesWithVotingMembersDown is ready 06:45:40,394 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} is now ready 06:45:40,394 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 06:45:40,394 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} is now ready 06:45:40,394 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-98551951] 06:45:40,394 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:45:40,394 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#491746921] 06:45:40,394 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#491746921] 06:45:40,396 PM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 06:45:40,396 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,396 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,396 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 06:45:40,396 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 06:45:40,396 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 06:45:40,396 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 06:45:40,396 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 06:45:40,396 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 06:45:40,396 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:45:40,396 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 06:45:40,398 PM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,398 PM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,398 PM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,398 PM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,398 PM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,398 PM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard cars 06:45:40,398 PM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 06:45:40,398 PM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 06:45:40,399 PM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,399 PM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard people 06:45:40,399 PM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 06:45:40,399 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,399 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard people 06:45:40,399 PM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 06:45:40,399 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 06:45:40,399 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 06:45:40,400 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,400 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard cars 06:45:40,400 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 06:45:40,400 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 06:45:40,400 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-98551951] 06:45:40,401 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:45:40,401 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#491746921] 06:45:40,405 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#491746921] 06:45:40,405 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-98551951] 06:45:40,405 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 06:45:40,405 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-1489846017] 06:45:40,405 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-1489846017] 06:45:40,405 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#1892699574] 06:45:40,405 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 06:45:40,406 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1438027026] 06:45:40,406 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1438027026] 06:45:40,406 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#1892699574] 06:45:40,406 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 06:45:40,406 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#1720862234] 06:45:40,406 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#1720862234] 06:45:40,406 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#74124179] 06:45:40,406 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 06:45:40,407 PM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1762011251] 06:45:40,407 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1762011251] 06:45:40,407 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#74124179] 06:45:40,407 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 06:45:40,407 PM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#1255208011] 06:45:40,407 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#1255208011] 06:45:40,407 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#1582585787] 06:45:40,407 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 06:45:40,407 PM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#1707897977] 06:45:40,407 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#1707897977] 06:45:40,407 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#1582585787] 06:45:40,407 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 06:45:40,407 PM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-10053512] 06:45:40,407 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-10053512] 06:45:40,407 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-2041762846] 06:45:40,407 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 06:45:40,407 PM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1284935122] 06:45:40,408 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1284935122] 06:45:40,408 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-2041762846] 06:45:40,408 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 06:45:40,408 PM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-591932693] 06:45:40,408 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-591932693] 06:45:40,408 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1952346284] 06:45:40,408 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 06:45:40,408 PM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-18726096] 06:45:40,408 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-18726096] 06:45:40,408 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1952346284] 06:45:40,408 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 06:45:40,408 PM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-870006138] 06:45:40,408 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-870006138] 06:45:40,409 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-98551951] 06:45:40,409 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:45:40,409 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#491746921] 06:45:40,409 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#491746921] 06:45:40,409 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 06:45:40,410 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,410 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,410 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 06:45:40,410 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 06:45:40,410 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 06:45:40,411 PM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 06:45:40,411 PM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 06:45:40,411 PM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 06:45:40,411 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 06:45:40,411 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 06:45:40,412 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,412 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,412 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 06:45:40,412 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 06:45:40,412 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 06:45:40,412 PM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 06:45:40,412 PM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 06:45:40,412 PM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 06:45:40,412 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 06:45:40,413 PM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:45:40,413 PM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:45:40,417 PM [cluster-test-pekko.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:40,417 PM [cluster-test-pekko.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:40,417 PM [cluster-test-pekko.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:40,417 PM [cluster-test-pekko.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:40,419 PM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 06:45:40,429 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 06:45:40,430 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,430 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,430 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 06:45:40,430 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 06:45:40,430 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 06:45:40,431 PM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 06:45:40,432 PM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 06:45:40,432 PM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 06:45:40,433 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 06:45:40,433 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 06:45:40,434 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,434 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:40,434 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 06:45:40,434 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 06:45:40,434 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 06:45:40,434 PM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 06:45:40,434 PM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 06:45:40,435 PM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 06:45:40,435 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 06:45:40,435 PM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:45:40,436 PM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:45:41,8 PM [cluster-test-shard-dispatcher-14] [WARN] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 06:45:41,8 PM [cluster-test-shard-dispatcher-32] [WARN] RaftActorVotingConfigSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 06:45:41,18 PM [cluster-test-shard-dispatcher-32] [WARN] RaftActorVotingConfigSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 06:45:41,18 PM [cluster-test-shard-dispatcher-14] [WARN] RaftActorVotingConfigSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true}, serversVisited=[]] 06:45:41,169 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 06:45:41,429 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [2850266606539582722] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 06:45:41,449 PM [cluster-test-pekko.actor.default-dispatcher-32] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:41,449 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-9129418018723133592] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 06:45:41,450 PM [cluster-test-pekko.actor.default-dispatcher-32] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:41,451 PM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/07/2025 18:45:41.453] [cluster-test-pekko.actor.internal-dispatcher-8] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:45:41,457 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 06:45:41,457 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:41,457 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 06:45:41,457 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 06:45:41,458 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@754d95c0 06:45:41,458 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@e829f78 06:45:41,458 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4750752a 06:45:41,458 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4750752a 06:45:41,458 PM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 06:45:41,458 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6e36e6a4 06:45:41,458 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6e36e6a4 06:45:41,458 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 06:45:41,879 PM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 06:45:41,879 PM [cluster-test-shard-dispatcher-32] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 421.3 ms 06:45:41,879 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 06:45:41,880 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 06:45:41,880 PM [cluster-test-shard-dispatcher-32] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 06:45:41,988 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 06:45:41,988 PM [cluster-test-shard-dispatcher-19] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 530.9 ms 06:45:41,989 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 06:45:41,989 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 06:45:41,990 PM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 06:45:41,990 PM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 06:45:41,991 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 06:45:41,991 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 06:45:41,991 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:41,991 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 06:45:41,991 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 06:45:41,992 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@12693ff 06:45:41,992 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@17422004 06:45:41,992 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6fd020d4 06:45:41,992 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6fd020d4 06:45:41,992 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 06:45:41,992 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1ba5e494 06:45:41,992 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1ba5e494 06:45:41,992 PM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 06:45:42,598 PM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 06:45:42,599 PM [cluster-test-shard-dispatcher-15] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 607.1 ms 06:45:42,599 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 06:45:42,599 PM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 06:45:42,599 PM [cluster-test-shard-dispatcher-32] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 607.7 ms 06:45:42,599 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 06:45:42,600 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 06:45:42,600 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 06:45:42,600 PM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 06:45:42,600 PM [cluster-test-shard-dispatcher-32] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 06:45:42,601 PM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 06:45:42,601 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 06:45:42,602 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:45:42,602 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:45:42,616 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/07/2025 18:45:42.619] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:45:42,682 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 06:45:42,691 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [6715941689084632576] 06:45:42,692 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 06:45:42,695 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 06:45:42,695 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 06:45:42,700 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:45:42,700 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:45:42,708 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:42,709 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:42,711 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 06:45:42,711 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 06:45:42,711 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 06:45:42,713 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardLeaderReplica 06:45:42,720 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-553010291] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:42,720 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 06:45:42,721 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 06:45:42,722 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 06:45:42,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testRemoveShardLeaderReplica 06:45:42,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testRemoveShardLeaderReplica 06:45:42,722 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:42,722 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 06:45:42,722 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 06:45:42,725 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 06:45:42,726 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@33eacb14} 06:45:42,726 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} started 06:45:42,727 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:42,727 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testRemoveShardLeaderReplica 06:45:42,727 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:42,727 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 06:45:42,727 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 06:45:42,727 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica on shard member-1-shard-cars-config_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#907446135] 06:45:42,727 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testRemoveShardLeaderReplica 06:45:42,728 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 06:45:42,728 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 06:45:42,728 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica on shard member-1-shard-people-config_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica#-47356957] 06:45:42,728 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: Shard created, persistent : true 06:45:42,728 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica#-47356957] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:42,728 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:42,730 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardLeaderReplica 06:45:42,731 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Shard created, persistent : true 06:45:42,731 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#907446135] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:42,733 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica-notifier#227446765] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:42,733 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica-notifier#227446765 created and ready for shard:member-1-shard-people-config_testRemoveShardLeaderReplica 06:45:42,733 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-1-shard-people-config_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 06:45:42,735 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-people-config_testRemoveShardLeaderReplica: journal open: applyTo=0 06:45:42,736 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-config_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 06:45:42,736 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica-notifier#1297367492 created and ready for shard:member-1-shard-cars-config_testRemoveShardLeaderReplica 06:45:42,736 PM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica-notifier#1297367492] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:42,737 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testRemoveShardLeaderReplica: journal open: applyTo=0 06:45:42,738 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#548849624] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:42,738 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 06:45:42,738 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 06:45:42,739 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:42,740 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:42,740 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 06:45:42,740 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testRemoveShardLeaderReplica 06:45:42,741 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testRemoveShardLeaderReplica 06:45:42,742 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:42,742 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 06:45:42,742 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 06:45:42,742 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:42,742 PM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-people-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:42,747 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 06:45:42,747 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 06:45:42,748 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 06:45:42,748 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testRemoveShardLeaderReplica is initialized 06:45:42,748 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:42,748 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:42,748 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from null to Follower 06:45:42,748 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 06:45:42,748 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from null to Follower 06:45:42,748 PM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-people-config_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 06:45:42,748 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} started 06:45:42,748 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@555cd067} 06:45:42,748 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:42,748 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:42,748 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 06:45:42,748 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testRemoveShardLeaderReplica is initialized 06:45:42,748 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from null to Follower 06:45:42,749 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:42,749 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 06:45:42,749 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testRemoveShardLeaderReplica 06:45:42,749 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from null to Follower 06:45:42,749 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 06:45:42,749 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 06:45:42,749 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica on shard member-1-shard-cars-oper_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica#382979683] 06:45:42,749 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testRemoveShardLeaderReplica 06:45:42,749 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 06:45:42,749 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 06:45:42,749 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica on shard member-1-shard-people-oper_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica#-86292786] 06:45:42,750 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Shard created, persistent : true 06:45:42,750 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica#382979683] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:42,750 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica#-86292786] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:42,750 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: Shard created, persistent : true 06:45:42,750 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica-notifier#-393771692 created and ready for shard:member-1-shard-cars-oper_testRemoveShardLeaderReplica 06:45:42,750 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica-notifier#-740062883 created and ready for shard:member-1-shard-people-oper_testRemoveShardLeaderReplica 06:45:42,750 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica-notifier#-393771692] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:42,750 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica-notifier#-740062883] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:42,750 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 06:45:42,751 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testRemoveShardLeaderReplica: journal open: applyTo=0 06:45:42,752 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:42,752 PM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:42,754 PM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 06:45:42,754 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 06:45:42,754 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testRemoveShardLeaderReplica is initialized 06:45:42,754 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:42,754 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:42,754 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from null to Follower 06:45:42,755 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 06:45:42,755 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from null to Follower 06:45:42,756 PM [cluster-test-shard-dispatcher-20] [INFO] RaftActor - member-1-shard-people-oper_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 06:45:42,757 PM [cluster-test-shard-dispatcher-20] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testRemoveShardLeaderReplica: journal open: applyTo=0 06:45:42,758 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:42,758 PM [cluster-test-shard-dispatcher-20] [INFO] PekkoRecovery - member-1-shard-people-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:42,760 PM [cluster-test-shard-dispatcher-20] [INFO] PekkoRecovery - member-1-shard-people-oper_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 06:45:42,760 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:42,760 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:42,760 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from null to Follower 06:45:42,760 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 06:45:42,760 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testRemoveShardLeaderReplica is initialized 06:45:42,760 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 06:45:42,760 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from null to Follower 06:45:42,761 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 06:45:42,766 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [4012044442542305790] 06:45:42,767 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 06:45:42,769 PM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:45:42,769 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 06:45:42,770 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:45:42,770 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:45:42,781 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:42,782 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:42,784 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardLeaderReplica 06:45:42,789 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-1068953958] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:42,793 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 06:45:42,793 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 06:45:42,794 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 06:45:42,794 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveShardLeaderReplica 06:45:42,795 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveShardLeaderReplica 06:45:42,795 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 06:45:42,795 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} started 06:45:42,796 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:42,797 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:42,800 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardLeaderReplica 06:45:42,800 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@1e896759} 06:45:42,800 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:42,800 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveShardLeaderReplica 06:45:42,802 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveShardLeaderReplica 06:45:42,803 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-2112090489] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:42,803 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Shard created, persistent : true 06:45:42,804 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica-notifier#-1385624472 created and ready for shard:member-2-shard-cars-config_testRemoveShardLeaderReplica 06:45:42,804 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica-notifier#-1385624472] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:42,804 PM [cluster-test-shard-dispatcher-22] [INFO] RaftActor - member-2-shard-cars-config_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 06:45:42,806 PM [cluster-test-shard-dispatcher-22] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveShardLeaderReplica: journal open: applyTo=0 06:45:42,807 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica#1910526759] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:42,807 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: Shard created, persistent : true 06:45:42,807 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#-306738158] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:42,808 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 06:45:42,808 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 06:45:42,808 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:42,809 PM [cluster-test-shard-dispatcher-21] [INFO] PekkoRecovery - member-2-shard-cars-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:42,809 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica-notifier#-1912181380 created and ready for shard:member-2-shard-people-config_testRemoveShardLeaderReplica 06:45:42,809 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica-notifier#-1912181380] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:42,809 PM [cluster-test-shard-dispatcher-22] [INFO] RaftActor - member-2-shard-people-config_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 06:45:42,810 PM [cluster-test-shard-dispatcher-22] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveShardLeaderReplica: journal open: applyTo=0 06:45:42,811 PM [cluster-test-shard-dispatcher-21] [INFO] PekkoRecovery - member-2-shard-cars-config_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 06:45:42,811 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:42,811 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:42,811 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , received role change from null to Follower 06:45:42,812 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 06:45:42,812 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveShardLeaderReplica is initialized 06:45:42,812 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 06:45:42,813 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardLeaderReplica from null to Follower 06:45:42,815 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 06:45:42,815 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} started 06:45:42,819 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 06:45:42,819 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveShardLeaderReplica 06:45:42,819 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:42,819 PM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-2-shard-people-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:42,819 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveShardLeaderReplica 06:45:42,820 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@1fca4af9} 06:45:42,820 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:42,820 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveShardLeaderReplica 06:45:42,820 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Shard created, persistent : true 06:45:42,821 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica#727907539] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:42,821 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica-notifier#-2095078327 created and ready for shard:member-2-shard-cars-oper_testRemoveShardLeaderReplica 06:45:42,821 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 06:45:42,821 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica-notifier#-2095078327] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:42,822 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveShardLeaderReplica: journal open: applyTo=0 06:45:42,823 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveShardLeaderReplica 06:45:42,824 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: Shard created, persistent : true 06:45:42,824 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica#1802718971] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:42,824 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica-notifier#71204998 created and ready for shard:member-2-shard-people-oper_testRemoveShardLeaderReplica 06:45:42,824 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-2-shard-people-oper_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 06:45:42,825 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica-notifier#71204998] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:42,825 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:42,825 PM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:42,822 PM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-2-shard-people-config_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 06:45:42,826 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:42,826 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:42,826 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardLeaderReplica , received role change from null to Follower 06:45:42,826 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 06:45:42,826 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveShardLeaderReplica is initialized 06:45:42,826 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 06:45:42,826 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveShardLeaderReplica from null to Follower 06:45:42,827 PM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 06:45:42,827 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:42,827 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:42,828 PM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardLeaderReplica , received role change from null to Follower 06:45:42,829 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveShardLeaderReplica: journal open: applyTo=0 06:45:42,830 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:42,831 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-2-shard-people-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:42,831 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 06:45:42,831 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveShardLeaderReplica is initialized 06:45:42,831 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 06:45:42,831 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveShardLeaderReplica from null to Follower 06:45:42,833 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-2-shard-people-oper_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 06:45:42,833 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:42,833 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:42,833 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardLeaderReplica , received role change from null to Follower 06:45:42,833 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 06:45:42,833 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveShardLeaderReplica is initialized 06:45:42,833 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 06:45:42,833 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveShardLeaderReplica from null to Follower 06:45:42,853 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 06:45:42,865 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 06:45:42,869 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [771727305118838944] 06:45:42,870 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 06:45:42,870 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 06:45:42,870 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:42,870 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 06:45:42,870 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 06:45:42,870 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 06:45:42,870 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 06:45:42,870 PM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 06:45:42,870 PM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 06:45:42,870 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:42,870 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 06:45:42,870 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 06:45:42,870 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 06:45:42,870 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 06:45:42,870 PM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-2-shard-people-config_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 06:45:42,870 PM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 06:45:42,872 PM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:45:42,872 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 06:45:42,873 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:45:42,873 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:45:42,885 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:42,887 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:42,888 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardLeaderReplica 06:45:42,895 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 06:45:42,895 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 06:45:42,895 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-2092524323] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:42,898 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 06:45:42,899 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testRemoveShardLeaderReplica 06:45:42,899 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testRemoveShardLeaderReplica 06:45:42,899 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 06:45:42,900 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} started 06:45:42,900 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:42,902 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:42,904 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardLeaderReplica 06:45:42,905 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@24a35d96} 06:45:42,905 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:42,905 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testRemoveShardLeaderReplica 06:45:42,905 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 06:45:42,905 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 06:45:42,905 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testRemoveShardLeaderReplica 06:45:42,905 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Shard created, persistent : true 06:45:42,906 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#-918887072] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:42,906 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#-1856311110] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:42,906 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-3-shard-cars-config_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 06:45:42,906 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica-notifier#246313015 created and ready for shard:member-3-shard-cars-config_testRemoveShardLeaderReplica 06:45:42,906 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica-notifier#246313015] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:42,907 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testRemoveShardLeaderReplica: journal open: applyTo=0 06:45:42,907 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: Shard created, persistent : true 06:45:42,907 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica#-1719515541] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:42,908 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-3-shard-people-config_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 06:45:42,908 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica-notifier#1544001525] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:42,908 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica-notifier#1544001525 created and ready for shard:member-3-shard-people-config_testRemoveShardLeaderReplica 06:45:42,909 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-3-shard-people-config_testRemoveShardLeaderReplica: journal open: applyTo=0 06:45:42,910 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:42,910 PM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-3-shard-cars-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:42,911 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:42,911 PM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-3-shard-people-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:42,912 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 06:45:42,912 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testRemoveShardLeaderReplica 06:45:42,912 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testRemoveShardLeaderReplica 06:45:42,913 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 06:45:42,914 PM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-3-shard-people-config_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 06:45:42,914 PM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-3-shard-cars-config_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 06:45:42,914 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 06:45:42,914 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testRemoveShardLeaderReplica is initialized 06:45:42,914 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 06:45:42,914 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testRemoveShardLeaderReplica is initialized 06:45:42,914 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:42,914 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:42,914 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:42,914 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:42,915 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , received role change from null to Follower 06:45:42,915 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 06:45:42,915 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardLeaderReplica , received role change from null to Follower 06:45:42,915 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 06:45:42,915 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} started 06:45:42,915 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardLeaderReplica from null to Follower 06:45:42,915 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testRemoveShardLeaderReplica from null to Follower 06:45:42,915 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@12c673d4} 06:45:42,915 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:42,915 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testRemoveShardLeaderReplica 06:45:42,915 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 06:45:42,916 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testRemoveShardLeaderReplica 06:45:42,916 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Shard created, persistent : true 06:45:42,916 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica#-1041439518] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:42,916 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 06:45:42,916 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica-notifier#1564101001 created and ready for shard:member-3-shard-cars-oper_testRemoveShardLeaderReplica 06:45:42,917 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: Shard created, persistent : true 06:45:42,917 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica-notifier#1564101001] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:42,917 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica#2046808346] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:42,917 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testRemoveShardLeaderReplica: journal open: applyTo=0 06:45:42,917 PM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica-notifier#646562628 created and ready for shard:member-3-shard-people-oper_testRemoveShardLeaderReplica 06:45:42,917 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica-notifier#646562628] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:42,917 PM [cluster-test-shard-dispatcher-28] [INFO] RaftActor - member-3-shard-people-oper_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 06:45:42,918 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:42,918 PM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:42,919 PM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testRemoveShardLeaderReplica: journal open: applyTo=0 06:45:42,920 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 06:45:42,920 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:42,921 PM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 06:45:42,921 PM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-people-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:42,921 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 06:45:42,921 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testRemoveShardLeaderReplica is initialized 06:45:42,921 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:42,921 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:42,921 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardLeaderReplica , received role change from null to Follower 06:45:42,921 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 06:45:42,921 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testRemoveShardLeaderReplica from null to Follower 06:45:42,922 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 06:45:42,923 PM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-people-oper_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 06:45:42,923 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:42,923 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 06:45:42,923 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 06:45:42,923 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 06:45:42,923 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 06:45:42,923 PM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-3-shard-cars-config_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 06:45:42,923 PM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-3-shard-people-config_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 06:45:42,923 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:42,923 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:42,923 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:42,923 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 06:45:42,923 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 06:45:42,923 PM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 06:45:42,923 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardLeaderReplica , received role change from null to Follower 06:45:42,928 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 06:45:42,928 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 06:45:42,928 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 06:45:42,928 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testRemoveShardLeaderReplica is initialized 06:45:42,928 PM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-3-shard-people-oper_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 06:45:42,928 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 06:45:42,928 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testRemoveShardLeaderReplica from null to Follower 06:45:43,71 PM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 06:45:43,71 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:45:43,71 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:43,71 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:43,71 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 06:45:43,71 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from Follower to Candidate 06:45:43,80 PM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 06:45:43,80 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:45:43,80 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:43,80 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:43,80 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 06:45:43,80 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from Follower to Candidate 06:45:43,89 PM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 06:45:43,89 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:45:43,89 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:43,89 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:43,89 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Follower to Candidate 06:45:43,89 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Follower to Candidate 06:45:43,99 PM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 06:45:43,99 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:45:43,99 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:43,99 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:43,99 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Follower to Candidate 06:45:43,99 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Follower to Candidate 06:45:43,430 PM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 06:45:43,481 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 06:45:43,482 PM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 06:45:43,509 PM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 06:45:43,707 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 06:45:43,707 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 06:45:43,708 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:43,708 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 06:45:43,708 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 06:45:43,709 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 06:45:43,709 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 06:45:43,709 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:43,709 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 06:45:43,709 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:43,709 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 06:45:43,709 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 06:45:43,709 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 06:45:43,709 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 06:45:43,709 PM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-1-shard-cars-config_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 06:45:43,709 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 06:45:43,709 PM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-1-shard-cars-config_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 06:45:43,709 PM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-1-shard-people-config_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 06:45:43,709 PM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-1-shard-people-config_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 06:45:43,709 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 06:45:43,709 PM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 06:45:43,709 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 06:45:43,709 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:43,709 PM [cluster-test-shard-dispatcher-19] [INFO] PeerInfos - member-1-shard-people-oper_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 06:45:43,709 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 06:45:43,709 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 06:45:43,709 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 06:45:43,709 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 06:45:43,709 PM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-people-oper_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 06:45:43,709 PM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 06:45:43,716 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:43,716 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 06:45:43,716 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 06:45:43,716 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 06:45:43,716 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 06:45:43,716 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:43,716 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 06:45:43,716 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 06:45:43,716 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 06:45:43,716 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 06:45:43,716 PM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 06:45:43,716 PM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 06:45:43,716 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:43,716 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 06:45:43,716 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 06:45:43,716 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 06:45:43,716 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 06:45:43,716 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:43,716 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 06:45:43,716 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 06:45:43,716 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 06:45:43,716 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 06:45:43,716 PM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 06:45:43,716 PM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-2-shard-people-config_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 06:45:43,796 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:43,796 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 06:45:43,796 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 06:45:43,796 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 06:45:43,796 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 06:45:43,797 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:43,797 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 06:45:43,797 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 06:45:43,797 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 06:45:43,797 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 06:45:43,797 PM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 06:45:43,797 PM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-3-shard-people-oper_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 06:45:43,797 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:43,797 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 06:45:43,797 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 06:45:43,798 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 06:45:43,798 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 06:45:43,798 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:43,798 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 06:45:43,798 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 06:45:43,798 PM [cluster-test-shard-dispatcher-14] [INFO] PeerInfos - member-3-shard-cars-config_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 06:45:43,798 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 06:45:43,798 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 06:45:43,798 PM [cluster-test-shard-dispatcher-14] [INFO] PeerInfos - member-3-shard-people-config_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 06:45:43,812 PM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 06:45:43,813 PM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:43,813 PM [cluster-test-shard-dispatcher-32] [INFO] Follower - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:43,821 PM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 06:45:43,821 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 06:45:43,821 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 06:45:43,821 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 06:45:43,821 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1669668c 06:45:43,821 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from Candidate to Leader 06:45:43,822 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardLeaderReplica status sync done false 06:45:43,822 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@39331484 06:45:43,822 PM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 06:45:43,823 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done false 06:45:43,823 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7855b506 06:45:43,823 PM [cluster-test-shard-dispatcher-32] [INFO] Follower - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:43,824 PM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:43,830 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 06:45:43,830 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 06:45:43,831 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@18ae9a31 06:45:43,831 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: became leader with frontend state for [] 06:45:43,831 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Candidate to Leader 06:45:43,831 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Candidate to Leader 06:45:43,832 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardLeaderReplica status sync done false 06:45:43,832 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3528d36d 06:45:43,832 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveShardLeaderReplica status sync done false 06:45:43,832 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7692547c 06:45:43,849 PM [cluster-test-shard-dispatcher-19] [INFO] Candidate - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 06:45:43,851 PM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:43,851 PM [cluster-test-shard-dispatcher-32] [INFO] Follower - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:43,857 PM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 06:45:43,857 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 06:45:43,857 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 06:45:43,857 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 06:45:43,857 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1c910dfc 06:45:43,857 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from Candidate to Leader 06:45:43,857 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 06:45:43,857 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} is now ready 06:45:43,857 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-553010291] 06:45:43,858 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:45:43,858 PM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#907446135] 06:45:43,858 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#907446135] 06:45:43,858 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-1068953958] 06:45:43,858 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 06:45:43,858 PM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-2112090489] 06:45:43,858 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-2112090489] 06:45:43,858 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-2092524323] 06:45:43,858 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 06:45:43,858 PM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#-1856311110] 06:45:43,858 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardLeaderReplica status sync done false 06:45:43,859 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#-1856311110] 06:45:43,859 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5a0945cd 06:45:43,859 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 06:45:43,859 PM [main] [INFO] ClusterAdminRpcService - Removing replica for shard ShardName{value=cars} memberName MemberName{value=member-1}, datastoreType Config 06:45:43,859 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-1}] 06:45:43,859 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 06:45:43,859 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: LocalPrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#907446135, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6132ef8] 06:45:43,860 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#907446135 for shard member-1-shard-cars-config_testRemoveShardLeaderReplica 06:45:43,860 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2a64a772 06:45:43,860 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 06:45:43,861 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveShardLeaderReplica} for shard cars 06:45:43,861 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Leader shard successfully removed the replica shard cars 06:45:43,861 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] ClusterAdminRpcService - Successfully removed replica for shard ShardName{value=cars} 06:45:43,861 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-1068953958] 06:45:43,862 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 06:45:43,862 PM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-2112090489] 06:45:43,862 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-2112090489] 06:45:43,862 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveShardLeaderReplica status sync done false 06:45:43,866 PM [cluster-test-shard-dispatcher-19] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]} has been successfully replicated to a majority of followers 06:45:43,867 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#907446135] with 10000 ms timeout 06:45:43,867 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1 : Local Shard replica for shard cars has been removed 06:45:43,867 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: persisting the shard list [people] 06:45:43,867 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardLeaderReplica status sync done true 06:45:43,868 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done true 06:45:43,869 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@57c4efe8 06:45:43,869 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7ceab312 06:45:43,869 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - No shard found with member Id member-1-shard-cars-config_testRemoveShardLeaderReplica 06:45:43,869 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@328a958d 06:45:43,869 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1baf4234 06:45:43,869 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testRemoveShardLeaderReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@328a958d 06:45:43,869 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: pauseLeader successfully completed - doing transfer 06:45:43,873 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: leader transfer complete - waiting for new leader 06:45:43,875 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 4 06:45:43,875 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 4 06:45:43,875 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:43,875 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:43,875 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 06:45:43,875 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3a99e2f1 06:45:43,875 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardLeaderReplica from Follower to Candidate 06:45:43,876 PM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Term 4 in "RequestVote{term=4, candidateId=member-2-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3}" message is greater than follower's term 3 - updating term 06:45:43,880 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 06:45:43,880 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 06:45:43,880 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 06:45:43,881 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 06:45:43,881 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@49d9fd74 06:45:43,881 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardLeaderReplica from Candidate to Leader 06:45:43,881 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 06:45:43,881 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done false 06:45:43,881 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2d638106 06:45:43,881 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 06:45:43,884 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done true 06:45:43,889 PM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 06:45:43,915 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-2112090489] 06:45:43,915 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-1068953958] 06:45:43,915 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 06:45:43,915 PM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-2112090489] 06:45:43,915 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-2112090489] 06:45:43,915 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-2092524323] 06:45:43,915 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 06:45:43,915 PM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#-1856311110] 06:45:43,915 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#-1856311110] 06:45:43,916 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-553010291] 06:45:43,916 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:45:43,916 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Local shard cars not found - shards present: [people] 06:45:43,916 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 06:45:43,917 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testRemoveShardLeaderReplica 06:45:43,917 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testRemoveShardLeaderReplica 06:45:43,917 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 06:45:43,917 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Stopping Shard 06:45:43,917 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: Stopping Shard 06:45:43,918 PM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testRemoveShardLeaderReplica: journal closed 06:45:43,918 PM [cluster-test-shard-dispatcher-32] [INFO] EnabledRaftStorage - member-3-shard-people-config_testRemoveShardLeaderReplica: journal closed 06:45:43,918 PM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 06:45:43,918 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 06:45:43,919 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 06:45:43,919 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testRemoveShardLeaderReplica 06:45:43,919 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testRemoveShardLeaderReplica 06:45:43,919 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 06:45:43,920 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Stopping Shard 06:45:43,920 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: Stopping Shard 06:45:43,920 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testRemoveShardLeaderReplica: journal closed 06:45:43,920 PM [cluster-test-shard-dispatcher-32] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testRemoveShardLeaderReplica: journal closed 06:45:43,920 PM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 06:45:43,920 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 06:45:43,922 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:45:43,922 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:45:43,927 PM [cluster-test-pekko.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:43,927 PM [cluster-test-pekko.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:43,927 PM [cluster-test-pekko.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:43,927 PM [cluster-test-pekko.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:43,929 PM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 06:45:43,932 PM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 06:45:43,933 PM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:43,939 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 06:45:43,939 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 06:45:43,939 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: became leader with frontend state for [] 06:45:43,939 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Candidate to Leader 06:45:43,940 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@29f2fe04 06:45:43,940 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Candidate to Leader 06:45:43,940 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 06:45:43,942 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 06:45:43,942 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardLeaderReplica status sync done false 06:45:43,942 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testRemoveShardLeaderReplica 06:45:43,942 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveShardLeaderReplica 06:45:43,942 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 06:45:43,943 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@eb862df 06:45:43,943 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-config_testRemoveShardLeaderReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@eb862df 06:45:43,943 PM [cluster-test-shard-dispatcher-26] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testRemoveShardLeaderReplica: pauseLeader successfully completed - doing transfer 06:45:43,943 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: Stopping Shard 06:45:43,943 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@59eefeaf 06:45:43,943 PM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveShardLeaderReplica: journal closed 06:45:43,943 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 06:45:43,943 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3831a944 06:45:44,148 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardLeaderReplica status sync done true 06:45:44,149 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 06:45:44,259 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardLeaderReplica status sync done true 06:45:44,260 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : -1, commitIndex : -1 06:45:44,459 PM [cluster-test-shard-dispatcher-26] [INFO] RaftActorBehavior - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Cannot append entries because sender's term 3 is less than 4 06:45:44,460 PM [cluster-test-shard-dispatcher-12] [INFO] AbstractLeader - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Term 4 in "AppendEntriesReply{term=4, success=false, followerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}" message is greater than leader's term 3 - switching to Follower 06:45:44,463 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to Follower, election term: 4 06:45:44,463 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 06:45:44,463 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:44,464 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from Leader to Follower 06:45:44,464 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@65af4e4 06:45:44,464 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - No shard found with member Id member-1-shard-cars-config_testRemoveShardLeaderReplica 06:45:44,464 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from Leader to Follower 06:45:44,469 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 06:45:44,578 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 06:45:44,728 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 06:45:44,790 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 06:45:44,900 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 06:45:44,937 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [771727305118838944] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 06:45:44,941 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [771727305118838944] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 06:45:45,110 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 06:45:45,218 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 06:45:45,429 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 06:45:45,540 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 06:45:45,750 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 06:45:45,859 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 06:45:45,887 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: leader not elected in time 06:45:45,887 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: Successfully transferred leadership to null in 2.020 s 06:45:45,887 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Stopping Shard 06:45:45,888 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testRemoveShardLeaderReplica: journal closed 06:45:45,888 PM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#907446135] 06:45:46,70 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 06:45:46,179 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 06:45:46,389 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 06:45:46,500 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 06:45:46,710 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 06:45:46,819 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 06:45:47,29 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 06:45:47,140 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 06:45:47,351 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 06:45:47,459 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 06:45:47,670 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 06:45:47,779 PM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 06:45:47,989 PM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 06:45:48,100 PM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 06:45:48,310 PM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 06:45:48,419 PM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 06:45:48,629 PM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 06:45:48,739 PM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 06:45:48,947 PM [cluster-test-shard-dispatcher-20] [WARN] Leader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 06:45:48,947 PM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 06:45:48,947 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 06:45:48,947 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 06:45:48,947 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 06:45:48,949 PM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 06:45:48,949 PM [cluster-test-shard-dispatcher-20] [INFO] IsolatedLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica: IsolatedLeader member-1-shard-people-oper_testRemoveShardLeaderReplica switching from IsolatedLeader to Leader 06:45:48,950 PM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 06:45:48,950 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 06:45:48,950 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from IsolatedLeader to Leader 06:45:48,950 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from IsolatedLeader to Leader 06:45:48,950 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 06:45:48,977 PM [cluster-test-shard-dispatcher-20] [WARN] Leader - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-people-config_testRemoveShardLeaderReplica from Leader to IsolatedLeader 06:45:48,977 PM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 06:45:48,977 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 06:45:48,978 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 06:45:48,978 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from Leader to IsolatedLeader 06:45:49,57 PM [cluster-test-shard-dispatcher-20] [WARN] Leader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 06:45:49,57 PM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 06:45:49,57 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 06:45:49,57 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 06:45:49,57 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 06:45:49,59 PM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 06:45:49,59 PM [cluster-test-shard-dispatcher-20] [INFO] IsolatedLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica: IsolatedLeader member-1-shard-cars-oper_testRemoveShardLeaderReplica switching from IsolatedLeader to Leader 06:45:49,59 PM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 06:45:49,59 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 06:45:49,59 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from IsolatedLeader to Leader 06:45:49,59 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from IsolatedLeader to Leader 06:45:49,59 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 06:45:49,269 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 06:45:49,379 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 06:45:49,589 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 06:45:49,699 PM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 06:45:49,827 PM [cluster-test-pekko.actor.default-dispatcher-13] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Up)]. 06:45:49,828 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:49,828 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 06:45:49,828 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:49,902 PM [cluster-test-pekko.actor.default-dispatcher-15] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Up)]. 06:45:49,903 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:49,903 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:49,910 PM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 06:45:50,20 PM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 06:45:50,229 PM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 06:45:50,338 PM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 06:45:50,550 PM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 06:45:50,659 PM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 06:45:50,869 PM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 06:45:50,978 PM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 06:45:51,189 PM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 06:45:51,299 PM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 06:45:51,510 PM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 06:45:51,559 PM [cluster-test-pekko.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 06:45:51,623 PM [cluster-test-pekko.actor.default-dispatcher-33] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 06:45:51,624 PM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 324, lastApplied : -1, commitIndex : -1 06:45:51,830 PM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 06:45:51,939 PM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 315, lastApplied : -1, commitIndex : -1 06:45:52,150 PM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 06:45:52,259 PM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 06:45:52,471 PM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 06:45:52,579 PM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 06:45:52,790 PM [cluster-test-shard-dispatcher-18] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 06:45:52,898 PM [cluster-test-shard-dispatcher-18] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 06:45:53,109 PM [cluster-test-shard-dispatcher-18] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 06:45:53,152 PM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testRemoveShardLeaderReplica: leader transfer aborted 06:45:53,152 PM [cluster-test-shard-dispatcher-25] [WARN] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testRemoveShardLeaderReplica: Failed to transfer leadership in 9.209 s 06:45:53,152 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: In unpauseLeader 06:45:53,152 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Stopping Shard 06:45:53,153 PM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveShardLeaderReplica: journal closed 06:45:53,153 PM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 06:45:53,153 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 06:45:53,154 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 06:45:53,154 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveShardLeaderReplica 06:45:53,154 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveShardLeaderReplica 06:45:53,154 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 06:45:53,155 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Stopping Shard 06:45:53,155 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: Stopping Shard 06:45:53,155 PM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveShardLeaderReplica: journal closed 06:45:53,155 PM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveShardLeaderReplica: journal closed 06:45:53,156 PM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 06:45:53,156 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 06:45:53,157 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:45:53,157 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:45:54,66 PM [cluster-test-shard-dispatcher-18] [WARN] Leader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 06:45:54,67 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 06:45:54,67 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 06:45:54,67 PM [cluster-test-pekko.actor.default-dispatcher-33] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 06:45:54,67 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 06:45:54,174 PM [cluster-test-pekko.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:54,174 PM [cluster-test-pekko.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:54,175 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 06:45:54,177 PM [cluster-test-pekko.actor.default-dispatcher-33] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [4012044442542305790] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated [WARN] [08/07/2025 18:45:54.178] [cluster-test-pekko.actor.internal-dispatcher-5] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:45:54,179 PM [cluster-test-shard-dispatcher-18] [WARN] Leader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 06:45:54,179 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 06:45:54,179 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 06:45:54,179 PM [cluster-test-pekko.actor.default-dispatcher-33] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 06:45:54,179 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 06:45:54,182 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 06:45:54,183 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testRemoveShardLeaderReplica 06:45:54,183 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 06:45:54,183 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: Stopping Shard 06:45:54,183 PM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-people-config_testRemoveShardLeaderReplica: journal closed 06:45:54,184 PM [cluster-test-pekko.actor.default-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 06:45:54,184 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 06:45:54,184 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 06:45:54,184 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testRemoveShardLeaderReplica 06:45:54,184 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testRemoveShardLeaderReplica 06:45:54,184 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 06:45:54,185 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Stopping Shard 06:45:54,185 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: Stopping Shard 06:45:54,185 PM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testRemoveShardLeaderReplica: journal closed 06:45:54,185 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testRemoveShardLeaderReplica: journal closed 06:45:54,185 PM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 06:45:54,185 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 06:45:54,186 PM [cluster-test-pekko.actor.default-dispatcher-33] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:45:54,186 PM [cluster-test-pekko.actor.default-dispatcher-33] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:45:54,187 PM [cluster-test-pekko.actor.default-dispatcher-33] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/07/2025 18:45:54.189] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:45:54,208 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 06:45:54,213 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-34776621191875652] 06:45:54,214 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 06:45:54,224 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 06:45:54,224 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 06:45:54,224 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:45:54,224 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:45:54,230 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:54,231 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:54,232 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 06:45:54,232 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 06:45:54,232 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 06:45:54,233 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,238 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard#1163159353] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,240 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 06:45:54,240 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 06:45:54,242 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 06:45:54,242 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,243 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,243 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:54,243 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,243 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,245 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 06:45:54,246 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@fdf10ec} 06:45:54,246 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} started 06:45:54,247 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:54,247 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,247 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:54,248 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:54,249 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,249 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,249 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#1957720274] 06:45:54,249 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,250 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,250 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,250 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard#-1722828169] 06:45:54,250 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Shard created, persistent : true 06:45:54,250 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#1957720274] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,250 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,252 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard#-1722828169] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,252 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard#2126308112] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,252 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Shard created, persistent : true 06:45:54,253 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 06:45:54,253 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 06:45:54,254 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 06:45:54,254 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,254 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,255 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:54,255 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,255 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,256 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#-1149061876 created and ready for shard:member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,257 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#12597628 created and ready for shard:member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,257 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#-1149061876] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,257 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Starting recovery with journal batch size 1 06:45:54,257 PM [cluster-test-shard-dispatcher-16] [INFO] RaftActor - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Starting recovery with journal batch size 1 06:45:54,258 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#12597628] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,258 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 06:45:54,259 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard, generation=0} started 06:45:54,259 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 06:45:54,259 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@723ff16d} 06:45:54,259 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:54,259 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,259 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: journal open: applyTo=0 06:45:54,261 PM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: journal open: applyTo=0 06:45:54,261 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,261 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,261 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard#-366019417] 06:45:54,261 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,261 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard#-366019417] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,261 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:54,261 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,261 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,261 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard#1266358450] 06:45:54,261 PM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:54,261 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard#1266358450] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,261 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Shard created, persistent : true 06:45:54,262 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard-notifier#685381032 created and ready for shard:member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,262 PM [cluster-test-shard-dispatcher-15] [INFO] RaftActor - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Starting recovery with journal batch size 1 06:45:54,262 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard-notifier#685381032] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,264 PM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: journal open: applyTo=0 06:45:54,264 PM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Local TermInfo store seeded with TermInfo{term=0} 06:45:54,264 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 06:45:54,264 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 06:45:54,264 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard is initialized 06:45:54,264 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:54,264 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 06:45:54,265 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Shard created, persistent : true 06:45:54,266 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard-notifier#2090612069 created and ready for shard:member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,266 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Starting recovery with journal batch size 1 06:45:54,266 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard-notifier#2090612069] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,267 PM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 06:45:54,267 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: journal open: applyTo=0 06:45:54,267 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:45:54,267 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 06:45:54,267 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:54,267 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 06:45:54,267 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 06:45:54,267 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 06:45:54,267 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 06:45:54,267 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 06:45:54,267 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,267 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6729ffd7 06:45:54,267 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 06:45:54,268 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:54,268 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:54,268 PM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:54,268 PM [cluster-test-shard-dispatcher-15] [INFO] PekkoRecovery - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:54,268 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:54,268 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:54,271 PM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Local TermInfo store seeded with TermInfo{term=0} 06:45:54,272 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 06:45:54,272 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:54,272 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 06:45:54,272 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 06:45:54,272 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard is initialized 06:45:54,272 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Local TermInfo store seeded with TermInfo{term=0} 06:45:54,272 PM [cluster-test-shard-dispatcher-15] [INFO] PekkoRecovery - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Local TermInfo store seeded with TermInfo{term=0} 06:45:54,273 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 06:45:54,273 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:54,273 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 06:45:54,273 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 06:45:54,273 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard is initialized 06:45:54,273 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 06:45:54,273 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:54,273 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 06:45:54,274 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 06:45:54,274 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard is initialized 06:45:54,275 PM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 06:45:54,275 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:45:54,275 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 06:45:54,275 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:54,275 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 06:45:54,275 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 06:45:54,275 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 06:45:54,275 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 06:45:54,275 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 06:45:54,275 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,275 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7ae6d5b3 06:45:54,275 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 06:45:54,275 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testChangeMemberVotingStatesForSingleNodeShard is ready 06:45:54,275 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} is now ready 06:45:54,276 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 06:45:54,276 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:45:54,276 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 06:45:54,276 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:54,277 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 06:45:54,277 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 06:45:54,277 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 06:45:54,277 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 06:45:54,277 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 06:45:54,277 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,278 PM [main] [INFO] ClusterAdminRpcService - Change member voting states for shard ShardName{value=cars}: {member-1=false} 06:45:54,280 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@514152f 06:45:54,280 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 06:45:54,280 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-1=false}] 06:45:54,280 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:45:54,281 PM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 06:45:54,281 PM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:45:54,281 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 06:45:54,281 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:54,281 PM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 06:45:54,281 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,281 PM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 06:45:54,281 PM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=INVALID_REQUEST, leaderHint=member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard} from local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,281 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 06:45:54,281 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 06:45:54,282 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 06:45:54,282 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,282 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@278b6ccd 06:45:54,282 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 06:45:54,282 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testChangeMemberVotingStatesForSingleNodeShard is ready 06:45:54,282 PM [cluster-test-pekko.actor.default-dispatcher-17] [ERROR] ClusterAdminRpcService - Failed to change member voting states for shard ShardName{value=cars} java.lang.IllegalArgumentException: The requested voting state change for shard cars is invalid. At least one member must be voting at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager$10.onComplete(ShardManager.java:1568) at org.apache.pekko.dispatch.OnComplete.internal(Future.scala:310) at org.apache.pekko.dispatch.OnComplete.internal(Future.scala:307) at org.apache.pekko.dispatch.japi$CallbackBridge.apply(Future.scala:234) at org.apache.pekko.dispatch.japi$CallbackBridge.apply(Future.scala:231) at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:484) at org.apache.pekko.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:73) at org.apache.pekko.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:110) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18) at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94) at org.apache.pekko.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:110) at org.apache.pekko.dispatch.TaskInvocation.run(AbstractDispatcher.scala:59) at org.apache.pekko.dispatch.ForkJoinExecutorConfigurator$PekkoForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:57) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 06:45:54,283 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard#1163159353] 06:45:54,283 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:45:54,283 PM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#1957720274] 06:45:54,283 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#1957720274] 06:45:54,283 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 06:45:54,284 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,284 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,284 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 06:45:54,284 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@14aaeb40 06:45:54,284 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@14aaeb40 06:45:54,284 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@7421249f 06:45:54,284 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@7421249f 06:45:54,284 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Stopping Shard 06:45:54,284 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Stopping Shard 06:45:54,285 PM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: journal closed 06:45:54,287 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: journal closed 06:45:54,287 PM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 06:45:54,287 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 06:45:54,287 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 06:45:54,289 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,289 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 06:45:54,289 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 06:45:54,289 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@3f9eb466 06:45:54,289 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@2a3c9013 06:45:54,289 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@3f9eb466 06:45:54,289 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@2a3c9013 06:45:54,289 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Stopping Shard 06:45:54,289 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Stopping Shard 06:45:54,290 PM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: journal closed 06:45:54,290 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: journal closed 06:45:54,291 PM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 06:45:54,291 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 06:45:54,291 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:45:54,291 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:45:54,292 PM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 06:45:54,306 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 06:45:54,317 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-3868205124738775821] 06:45:54,317 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 06:45:54,319 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 06:45:54,319 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 06:45:54,320 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:45:54,320 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:45:54,331 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:54,332 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:54,333 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 06:45:54,333 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 06:45:54,333 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 06:45:54,334 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatusForShard 06:45:54,337 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 06:45:54,337 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 06:45:54,338 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-236354691] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,341 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 06:45:54,342 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:54,342 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testChangeMemberVotingStatusForShard 06:45:54,342 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:54,342 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:54,342 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 06:45:54,343 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 06:45:54,344 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@2ec0f33a} 06:45:54,344 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} started 06:45:54,345 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:54,345 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:54,345 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:54,345 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:54,345 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:54,345 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard on shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-1139258471] 06:45:54,345 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testChangeMemberVotingStatusForShard 06:45:54,346 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-1139258471] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,346 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Shard created, persistent : true 06:45:54,346 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#2134116462 created and ready for shard:member-1-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:54,346 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#2134116462] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,346 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:54,346 PM [cluster-test-shard-dispatcher-17] [INFO] RaftActor - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 06:45:54,348 PM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: journal open: applyTo=0 06:45:54,348 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatusForShard 06:45:54,349 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Shard created, persistent : true 06:45:54,349 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard#-673600406] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,350 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard-notifier#642843456 created and ready for shard:member-1-shard-people-config_testChangeMemberVotingStatusForShard 06:45:54,350 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard-notifier#642843456] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,350 PM [cluster-test-shard-dispatcher-17] [INFO] RaftActor - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 06:45:54,350 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 06:45:54,350 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 06:45:54,350 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard on shard member-1-shard-people-config_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard#-673600406] 06:45:54,350 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:54,350 PM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-1-shard-people-config_testChangeMemberVotingStatusForShard: journal open: applyTo=0 06:45:54,350 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:54,351 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#687333590] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,352 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:54,352 PM [cluster-test-shard-dispatcher-20] [INFO] PekkoRecovery - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:54,352 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 06:45:54,352 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 06:45:54,353 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 06:45:54,353 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:54,353 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:54,353 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:54,353 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:54,353 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:54,354 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 06:45:54,354 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 06:45:54,354 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard is initialized 06:45:54,354 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 06:45:54,354 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:54,354 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 06:45:54,354 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 06:45:54,354 PM [cluster-test-shard-dispatcher-20] [INFO] PekkoRecovery - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 06:45:54,354 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 06:45:54,354 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatusForShard from null to Follower 06:45:54,355 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 06:45:54,355 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testChangeMemberVotingStatusForShard is initialized 06:45:54,355 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 06:45:54,355 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:54,355 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 06:45:54,355 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@613a12a9} 06:45:54,355 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 06:45:54,355 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatusForShard from null to Follower 06:45:54,355 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} started 06:45:54,355 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:54,355 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:54,356 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:54,356 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:54,356 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard#634465388] 06:45:54,356 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:54,357 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:54,357 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:54,357 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Shard created, persistent : true 06:45:54,357 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard on shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard#618749800] 06:45:54,357 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard#634465388] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,357 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard#618749800] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,357 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Shard created, persistent : true 06:45:54,357 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#632525576] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,357 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#632525576 created and ready for shard:member-1-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:54,357 PM [cluster-test-shard-dispatcher-20] [INFO] RaftActor - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 06:45:54,358 PM [cluster-test-shard-dispatcher-20] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: journal open: applyTo=0 06:45:54,361 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#-322634361 created and ready for shard:member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:54,361 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#-322634361] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,361 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 06:45:54,361 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:54,362 PM [cluster-test-shard-dispatcher-20] [INFO] PekkoRecovery - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:54,363 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: journal open: applyTo=0 06:45:54,364 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:54,364 PM [cluster-test-shard-dispatcher-20] [INFO] PekkoRecovery - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 06:45:54,364 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:54,364 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 06:45:54,364 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:54,364 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 06:45:54,364 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard is initialized 06:45:54,364 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 06:45:54,364 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 06:45:54,365 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatusForShard from null to Follower 06:45:54,365 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 06:45:54,366 PM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 06:45:54,366 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 06:45:54,366 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 06:45:54,366 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard is initialized 06:45:54,366 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:54,367 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 06:45:54,367 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 06:45:54,367 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard from null to Follower 06:45:54,375 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [7299321925510720773] 06:45:54,375 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 06:45:54,378 PM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:45:54,378 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 06:45:54,380 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:45:54,380 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:45:54,391 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:54,395 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:54,398 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatusForShard 06:45:54,406 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#181987901] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,406 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 06:45:54,407 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 06:45:54,415 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 06:45:54,415 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:54,415 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testChangeMemberVotingStatusForShard 06:45:54,416 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 06:45:54,418 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@10c66a19} 06:45:54,418 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} started 06:45:54,419 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:54,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:54,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:54,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testChangeMemberVotingStatusForShard 06:45:54,420 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:54,421 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Shard created, persistent : true 06:45:54,421 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#366895975] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,421 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 06:45:54,422 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#1844948793 created and ready for shard:member-2-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:54,422 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#1844948793] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,423 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: journal open: applyTo=0 06:45:54,423 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Shard created, persistent : true 06:45:54,423 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard#2019178937] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,423 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-1336406057 created and ready for shard:member-2-shard-people-config_testChangeMemberVotingStatusForShard 06:45:54,423 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-1336406057] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,424 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 06:45:54,425 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-people-config_testChangeMemberVotingStatusForShard: journal open: applyTo=0 06:45:54,428 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatusForShard 06:45:54,430 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 06:45:54,430 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 06:45:54,430 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:54,431 PM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:54,433 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:54,433 PM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:54,434 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#671167656] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,434 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 06:45:54,434 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:54,435 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:54,436 PM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 06:45:54,437 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 06:45:54,437 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard is initialized 06:45:54,437 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 06:45:54,437 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:54,437 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 06:45:54,437 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 06:45:54,437 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testChangeMemberVotingStatusForShard from null to Follower 06:45:54,438 PM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 06:45:54,438 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 06:45:54,438 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testChangeMemberVotingStatusForShard is initialized 06:45:54,438 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 06:45:54,439 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 06:45:54,439 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:54,439 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 06:45:54,439 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 06:45:54,439 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testChangeMemberVotingStatusForShard from null to Follower 06:45:54,439 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@31fd4036} 06:45:54,439 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} started 06:45:54,440 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:54,440 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:54,440 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:54,441 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Shard created, persistent : true 06:45:54,441 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Shard created, persistent : true 06:45:54,441 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 06:45:54,441 PM [cluster-test-shard-dispatcher-29] [INFO] RaftActor - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 06:45:54,442 PM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: journal open: applyTo=0 06:45:54,442 PM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: journal open: applyTo=0 06:45:54,443 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard#407704793] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,443 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard#-1250349921] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,443 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#-254867256 created and ready for shard:member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:54,443 PM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#-254867256] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,446 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:54,446 PM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:54,447 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:54,447 PM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:54,447 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 06:45:54,449 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 06:45:54,450 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:54,450 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:54,450 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:54,450 PM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 06:45:54,450 PM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 06:45:54,451 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 06:45:54,451 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:54,451 PM [cluster-test-shard-dispatcher-16] [INFO] PeerInfos - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:54,451 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:54,451 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:54,451 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:54,451 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:54,451 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:54,451 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 06:45:54,451 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:54,451 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 06:45:54,451 PM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:54,451 PM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:54,452 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#168161640 created and ready for shard:member-2-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:54,452 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 06:45:54,456 PM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#168161640] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,457 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 06:45:54,457 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 06:45:54,457 PM [cluster-test-shard-dispatcher-32] [INFO] PeerInfos - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 06:45:54,457 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 06:45:54,457 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard is initialized 06:45:54,457 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 06:45:54,457 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard is initialized 06:45:54,457 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 06:45:54,457 PM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 06:45:54,457 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testChangeMemberVotingStatusForShard from null to Follower 06:45:54,457 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard from null to Follower 06:45:54,460 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 06:45:54,466 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [4906408328718416941] 06:45:54,467 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 06:45:54,468 PM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:45:54,468 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 06:45:54,477 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:54,481 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:54,482 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatusForShard 06:45:54,484 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 06:45:54,484 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 06:45:54,488 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#711136995] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,488 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 06:45:54,489 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 06:45:54,497 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 06:45:54,497 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:54,498 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testChangeMemberVotingStatusForShard 06:45:54,500 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 06:45:54,500 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@350a55a5} 06:45:54,500 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} started 06:45:54,501 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:54,501 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:54,501 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:54,501 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testChangeMemberVotingStatusForShard 06:45:54,502 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Shard created, persistent : true 06:45:54,502 PM [cluster-test-shard-dispatcher-14] [INFO] RaftActor - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 06:45:54,502 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 06:45:54,502 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#-1650249622] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,502 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#-1343650029 created and ready for shard:member-3-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:54,503 PM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: journal open: applyTo=0 06:45:54,503 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Shard created, persistent : true 06:45:54,503 PM [cluster-test-shard-dispatcher-14] [INFO] RaftActor - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 06:45:54,504 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard-notifier#691000746 created and ready for shard:member-3-shard-people-config_testChangeMemberVotingStatusForShard 06:45:54,504 PM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-3-shard-people-config_testChangeMemberVotingStatusForShard: journal open: applyTo=0 06:45:54,504 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 06:45:54,505 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatusForShard 06:45:54,507 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 06:45:54,509 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:54,509 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:54,509 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:54,509 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 06:45:54,509 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 06:45:54,509 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#-1343650029] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,509 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard#-1116926556] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,509 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard-notifier#691000746] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,510 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 06:45:54,510 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 06:45:54,510 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:54,510 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#-1911824107] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,511 PM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:54,513 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:54,513 PM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:54,513 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 06:45:54,513 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:54,514 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:54,514 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 06:45:54,514 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:54,514 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:54,514 PM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 06:45:54,514 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 06:45:54,514 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard is initialized 06:45:54,514 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 06:45:54,514 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:54,514 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 06:45:54,514 PM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:54,515 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 06:45:54,515 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testChangeMemberVotingStatusForShard from null to Follower 06:45:54,515 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 06:45:54,515 PM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 06:45:54,515 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 06:45:54,515 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testChangeMemberVotingStatusForShard is initialized 06:45:54,515 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 06:45:54,515 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:54,516 PM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 06:45:54,516 PM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 06:45:54,516 PM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 06:45:54,516 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} started 06:45:54,516 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testChangeMemberVotingStatusForShard from null to Follower 06:45:54,516 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f4ea87529d0@49268d47} 06:45:54,516 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 06:45:54,516 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 06:45:54,516 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:54,516 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:54,516 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:54,516 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard#-1094805783] 06:45:54,516 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:54,517 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:54,517 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard#-1094805783] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,517 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:54,517 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard on shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard#-658378065] 06:45:54,517 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Shard created, persistent : true 06:45:54,517 PM [cluster-test-shard-dispatcher-30] [INFO] RaftActor - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 06:45:54,518 PM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: journal open: applyTo=0 06:45:54,518 PM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard#-658378065] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,518 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#606471434] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,518 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#606471434 created and ready for shard:member-3-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:54,519 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Shard created, persistent : true 06:45:54,519 PM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#-1338001822 created and ready for shard:member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:54,519 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:54,519 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#-1338001822] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 06:45:54,519 PM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:54,520 PM [cluster-test-shard-dispatcher-29] [INFO] RaftActor - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 06:45:54,521 PM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: journal open: applyTo=0 06:45:54,521 PM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 06:45:54,522 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 06:45:54,522 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:54,522 PM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 06:45:54,522 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 06:45:54,522 PM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:54,522 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard is initialized 06:45:54,522 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 06:45:54,522 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testChangeMemberVotingStatusForShard from null to Follower 06:45:54,522 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 06:45:54,522 PM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 06:45:54,525 PM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 06:45:54,525 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 06:45:54,525 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:54,525 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 06:45:54,525 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 06:45:54,525 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard is initialized 06:45:54,526 PM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:54,526 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 06:45:54,526 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard from null to Follower 06:45:54,687 PM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 06:45:54,687 PM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:45:54,687 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 06:45:54,687 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:54,687 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 06:45:54,687 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard from Follower to Candidate 06:45:54,757 PM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 06:45:54,757 PM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:45:54,757 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 06:45:54,757 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:54,757 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 06:45:54,758 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatusForShard from Follower to Candidate 06:45:54,758 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 06:45:54,758 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:45:54,758 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 06:45:54,758 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:54,759 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 06:45:54,759 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatusForShard from Follower to Candidate 06:45:54,766 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 06:45:54,766 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 06:45:54,766 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 06:45:54,766 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 06:45:54,766 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 06:45:54,767 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatusForShard from Follower to Candidate 06:45:55,47 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 06:45:55,77 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 06:45:55,147 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 06:45:55,177 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 06:45:55,335 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 06:45:55,336 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 06:45:55,337 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:55,337 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:55,337 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:55,337 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 06:45:55,337 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 06:45:55,337 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:55,337 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:55,337 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:55,337 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 06:45:55,337 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 06:45:55,337 PM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:55,337 PM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:55,337 PM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 06:45:55,338 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:55,338 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:55,338 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:55,338 PM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 06:45:55,338 PM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:55,338 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:55,338 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:55,338 PM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:55,338 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:55,338 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:55,338 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:55,338 PM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:55,338 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:55,338 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:55,338 PM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:55,342 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:55,342 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:55,342 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:55,342 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:55,342 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:55,343 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:55,343 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:55,343 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:55,343 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:55,343 PM [cluster-test-shard-dispatcher-33] [INFO] PeerInfos - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:55,343 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:55,343 PM [cluster-test-shard-dispatcher-33] [INFO] PeerInfos - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:55,343 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:55,343 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:55,343 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:55,343 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 06:45:55,343 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 06:45:55,343 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:55,343 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:55,343 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:55,343 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 06:45:55,343 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 06:45:55,343 PM [cluster-test-shard-dispatcher-33] [INFO] PeerInfos - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 06:45:55,343 PM [cluster-test-shard-dispatcher-34] [INFO] PeerInfos - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:55,412 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:55,413 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:55,413 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:55,413 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:55,413 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:55,413 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:55,413 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:55,413 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:55,413 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:55,413 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:55,413 PM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:55,413 PM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:55,413 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 06:45:55,413 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:55,413 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:55,413 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 06:45:55,413 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 06:45:55,413 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 06:45:55,413 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:55,413 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:55,413 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 06:45:55,413 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 06:45:55,413 PM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:55,413 PM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 06:45:55,447 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 06:45:55,449 PM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:55,450 PM [cluster-test-shard-dispatcher-32] [INFO] Follower - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:55,457 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 06:45:55,457 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 06:45:55,457 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 06:45:55,457 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@47bb73b6 06:45:55,457 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 06:45:55,457 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard from Candidate to Leader 06:45:55,458 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@66386403 06:45:55,458 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard status sync done false 06:45:55,458 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard status sync done false 06:45:55,458 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7882fc5 06:45:55,484 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 06:45:55,486 PM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 06:45:55,488 PM [cluster-test-shard-dispatcher-34] [INFO] Follower - member-2-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:55,488 PM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-3-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:55,493 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 06:45:55,493 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 06:45:55,494 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 06:45:55,494 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 06:45:55,494 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@497c9531 06:45:55,494 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatusForShard from Candidate to Leader 06:45:55,494 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testChangeMemberVotingStatusForShard is ready 06:45:55,494 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testChangeMemberVotingStatusForShard status sync done false 06:45:55,494 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@68ff86b4 06:45:55,495 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testChangeMemberVotingStatusForShard is ready 06:45:55,495 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testChangeMemberVotingStatusForShard status sync done false 06:45:55,495 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@ac5a476 06:45:55,495 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testChangeMemberVotingStatusForShard is ready 06:45:55,557 PM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 06:45:55,558 PM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 06:45:55,559 PM [cluster-test-shard-dispatcher-34] [INFO] Follower - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:55,559 PM [cluster-test-shard-dispatcher-32] [INFO] Follower - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:55,561 PM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:55,561 PM [cluster-test-shard-dispatcher-31] [INFO] Follower - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 06:45:55,565 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 06:45:55,566 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 06:45:55,566 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 06:45:55,566 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 06:45:55,566 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4d4318c6 06:45:55,566 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatusForShard from Candidate to Leader 06:45:55,567 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testChangeMemberVotingStatusForShard status sync done false 06:45:55,567 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4eb51933 06:45:55,567 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testChangeMemberVotingStatusForShard status sync done false 06:45:55,567 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7f3d7f22 06:45:55,568 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 06:45:55,568 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 06:45:55,568 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 06:45:55,568 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 06:45:55,568 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@8419f0a 06:45:55,569 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatusForShard from Candidate to Leader 06:45:55,569 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testChangeMemberVotingStatusForShard is ready 06:45:55,569 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} is now ready 06:45:55,569 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 06:45:55,569 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatusForShard status sync done false 06:45:55,569 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatusForShard status sync done false 06:45:55,569 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@279fd4df 06:45:55,569 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6cc00f73 06:45:55,569 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testChangeMemberVotingStatusForShard is ready 06:45:55,569 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testChangeMemberVotingStatusForShard is ready 06:45:55,569 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} is now ready 06:45:55,569 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-236354691] 06:45:55,569 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:45:55,569 PM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-1139258471] 06:45:55,569 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-1139258471] 06:45:55,570 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#181987901] 06:45:55,570 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 06:45:55,570 PM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#366895975] 06:45:55,570 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#366895975] 06:45:55,570 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#711136995] 06:45:55,570 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 06:45:55,570 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#-1650249622] 06:45:55,570 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#-1650249622] 06:45:55,570 PM [main] [INFO] ClusterAdminRpcService - Change member voting states for shard ShardName{value=cars}: {member-2=false, member-3=false} 06:45:55,570 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-2=false, member-3=false}] 06:45:55,571 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 06:45:55,571 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatusForShard=false, member-3-shard-cars-config_testChangeMemberVotingStatusForShard=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:55,573 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]} has been successfully replicated to a majority of followers 06:45:55,576 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testChangeMemberVotingStatusForShard} from local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:55,576 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 06:45:55,576 PM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ClusterAdminRpcService - Successfully changed member voting states for shard ShardName{value=cars} 06:45:55,578 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-236354691] 06:45:55,578 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 06:45:55,578 PM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-1139258471] 06:45:55,578 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-1139258471] 06:45:55,578 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#181987901] 06:45:55,578 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 06:45:55,578 PM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#366895975] 06:45:55,578 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#366895975] 06:45:55,579 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatusForShard status sync done true 06:45:55,580 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#711136995] 06:45:55,580 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 06:45:55,580 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatusForShard status sync done true 06:45:55,580 PM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#-1650249622] 06:45:55,580 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#-1650249622] 06:45:55,580 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 06:45:55,580 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:55,580 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testChangeMemberVotingStatusForShard 06:45:55,581 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 06:45:55,581 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Stopping Shard 06:45:55,581 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Stopping Shard 06:45:55,581 PM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: journal closed 06:45:55,581 PM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-3-shard-people-config_testChangeMemberVotingStatusForShard: journal closed 06:45:55,581 PM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 06:45:55,581 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 06:45:55,582 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 06:45:55,582 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:55,582 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:55,582 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 06:45:55,582 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Stopping Shard 06:45:55,582 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Stopping Shard 06:45:55,582 PM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: journal closed 06:45:55,582 PM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: journal closed 06:45:55,582 PM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 06:45:55,582 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 06:45:55,583 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:45:55,583 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:45:55,588 PM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:55,588 PM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:55,588 PM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:55,589 PM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:55,591 PM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 06:45:55,604 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 06:45:55,604 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:55,605 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testChangeMemberVotingStatusForShard 06:45:55,605 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 06:45:55,605 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Stopping Shard 06:45:55,605 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Stopping Shard 06:45:55,607 PM [cluster-test-shard-dispatcher-32] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: journal closed 06:45:55,607 PM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-2-shard-people-config_testChangeMemberVotingStatusForShard: journal closed 06:45:55,607 PM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 06:45:55,607 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 06:45:55,607 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 06:45:55,608 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:55,608 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:55,608 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 06:45:55,608 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Stopping Shard 06:45:55,608 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Stopping Shard 06:45:55,608 PM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: journal closed 06:45:55,608 PM [cluster-test-shard-dispatcher-33] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: journal closed 06:45:55,608 PM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 06:45:55,609 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 06:45:55,610 PM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:45:55,610 PM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:45:56,354 PM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 06:45:56,604 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [4906408328718416941] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 06:45:56,624 PM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [7299321925510720773] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 06:45:56,631 PM [cluster-test-pekko.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:56,631 PM [cluster-test-pekko.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 06:45:56,633 PM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/07/2025 18:45:56.634] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 06:45:56,640 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 06:45:56,640 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard 06:45:56,640 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testChangeMemberVotingStatusForShard 06:45:56,640 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 06:45:56,641 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@686e6045 06:45:56,641 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5764b792 06:45:56,641 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5764b792 06:45:56,641 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3650133d 06:45:56,641 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 06:45:56,641 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3650133d 06:45:56,641 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 06:45:56,641 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@27011efc 06:45:57,164 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatusForShard: leader transfer aborted 06:45:57,164 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: leader transfer aborted 06:45:57,164 PM [cluster-test-shard-dispatcher-20] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 523.4 ms 06:45:57,164 PM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 523.4 ms 06:45:57,164 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: In unpauseLeader 06:45:57,164 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: In unpauseLeader 06:45:57,165 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Stopping Shard 06:45:57,165 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Stopping Shard 06:45:57,165 PM [cluster-test-shard-dispatcher-20] [INFO] EnabledRaftStorage - member-1-shard-people-config_testChangeMemberVotingStatusForShard: journal closed 06:45:57,165 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: journal closed 06:45:57,166 PM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 06:45:57,166 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 06:45:57,166 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 06:45:57,167 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 06:45:57,167 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard 06:45:57,167 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 06:45:57,167 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@32886d9 06:45:57,167 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@43408345 06:45:57,167 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@77d7c67d 06:45:57,167 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@43408345 06:45:57,167 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 06:45:57,168 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7211a315 06:45:57,168 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7211a315 06:45:57,168 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 06:45:57,694 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: leader transfer aborted 06:45:57,694 PM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 526.6 ms 06:45:57,694 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: In unpauseLeader 06:45:57,695 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Stopping Shard 06:45:57,695 PM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: journal closed 06:45:57,734 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: leader transfer aborted 06:45:57,734 PM [cluster-test-shard-dispatcher-20] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 566.8 ms 06:45:57,734 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: In unpauseLeader 06:45:57,735 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Stopping Shard 06:45:57,736 PM [cluster-test-shard-dispatcher-20] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: journal closed 06:45:57,736 PM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 06:45:57,736 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 06:45:57,737 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 06:45:57,737 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 06:45:57,740 PM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/07/2025 18:45:57.741] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config