07:01:58,518 AM [test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:01:58,532 AM [test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://test@127.0.0.1:2565] with UID [4474114977632101902] 07:01:58,532 AM [test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://test@127.0.0.1:2565] - Starting up, Akka version [2.6.21] ... 07:01:58,540 AM [test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://test@127.0.0.1:2565] - Registered cluster JMX MBean [akka:type=Cluster] 07:01:58,540 AM [test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://test@127.0.0.1:2565] - Started up successfully 07:01:58,544 AM [test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://test@127.0.0.1:2565] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 07:01:58,544 AM [test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://test@127.0.0.1:2565] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 07:01:58,656 AM [test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$a#-206135593] to Actor[akka://test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 07:01:58,659 AM [test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config1 07:01:58,661 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shard-manager-config1 07:01:58,661 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config1: Creating local shard: member-1-shard-default-config1 07:01:58,665 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:01:58,665 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config1 07:01:58,665 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config1: Initializing shard [default] 07:01:58,665 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-default-config1 is initialized 07:01:58,665 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config1: Received role changed for member-1-shard-default-config1 from null to Follower 07:01:58,666 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config1: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=default, meberVotingStatusMap={member-2=true}] 07:01:58,668 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config1: findLocalShard : default 07:01:58,669 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-default-config1=true}, serversVisited=[]] to local shard akka://test/user/member-1-shard-default-config1 07:01:58,671 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config1: Received ServerChangeReply{status=NO_LEADER} from local shard akka://test/user/member-1-shard-default-config1 07:01:58,671 AM [test-akka.actor.default-dispatcher-11] [WARN] ShardManager - shard-manager-config1: ChangeServersVotingStatus failed for shard default with status NO_LEADER 07:01:58,824 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-default-config1#1718328276] 07:01:58,839 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForShardLeader starting 07:01:58,841 AM [test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$b#-1111195863] to Actor[akka://test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 07:01:58,842 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config2 07:01:58,846 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config2 07:01:58,846 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config2: Creating local shard: member-1-shard-default-config2 07:01:58,852 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:01:58,852 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config2 07:01:58,852 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 07:01:58,852 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config2: Scheduling 10000 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config2, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 07:01:59,3 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config2: Initializing shard [default] 07:01:59,3 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-default-config2 is initialized 07:01:59,3 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-default-config2 is initialized - notifying 1 OnShardInitialized callbacks 07:01:59,153 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config2: Received role changed for member-1-shard-default-config2 from Candidate to Leader 07:01:59,153 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config2 is initialized - notifying 1 OnShardInitialized callbacks 07:01:59,303 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config2, leaderId=member-1-shard-default-config2, leaderPayloadVersion=13] 07:01:59,303 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config2 is ready - notifying 1 OnShardInitialized callbacks 07:01:59,303 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config2: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://test@127.0.0.1:2565/user/member-1-shard-default-config#1467652288, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$dQrdkTAT@54118bbf] 07:01:59,303 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config2: All Shards are ready - data store config2 is ready 07:01:59,504 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForShardLeader ending 07:01:59,605 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$b#-1111195863] 07:01:59,616 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config2 07:01:59,621 AM [test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://test/user/$$b] to Actor[akka://test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 07:01:59,621 AM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config3 07:01:59,624 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config3 07:01:59,624 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config3: Creating local shard: member-1-shard-default-config3 07:01:59,625 AM [main] [INFO] ShardManager - shard-manager-config3: Received role changed for member-1-shard-default-config3 from Follower to Candidate 07:01:59,625 AM [main] [INFO] ShardManager - shard-manager-config3 Received follower initial sync status for member-1-shard-default-config3 status sync done true 07:01:59,727 AM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$b] 07:01:59,738 AM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config3 07:01:59,750 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:01:59,764 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-8253899696999964019] 07:01:59,764 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 07:01:59,772 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:01:59,772 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 07:01:59,777 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 07:01:59,777 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 07:01:59,786 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 07:01:59,788 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config4] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 07:01:59,791 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config4 07:01:59,795 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 07:01:59,795 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 07:01:59,797 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 07:01:59,805 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config4 07:01:59,805 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config4: Creating local shard: member-1-shard-default-config4 07:01:59,805 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config4: Creating local shard: member-1-shard-astronauts-config4 07:01:59,805 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config4: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 07:01:59,805 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-default-config4 07:01:59,805 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-astronauts-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-astronauts-config4 07:01:59,810 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-1648250021952306508] 07:01:59,810 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 07:01:59,812 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:01:59,812 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 07:01:59,815 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 07:01:59,815 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 07:01:59,822 AM [main] [ERROR] ShardManagerTest - Mock Shard Leader Actor : TestActor[akka://cluster-test/user/member-2-shard-default-config4] 07:01:59,825 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config4] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 07:01:59,827 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config4 07:01:59,827 AM [main] [ERROR] ShardManagerTest - Forwarding actor : Actor[akka://cluster-test/user/shardmanager-config4/member-2-shard-default-config4#-621627318] 07:01:59,832 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config4 07:01:59,832 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config4: Creating local shard: member-2-shard-default-config4 07:01:59,838 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:01:59,838 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - Creating Shard member-2-shard-default-config4 07:01:59,838 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config4: Initializing shard [default] 07:01:59,838 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardInformation - Shard member-2-shard-default-config4 is initialized 07:01:59,838 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ShardManager - shard-manager-config4: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-config4, leaderId=member-2-shard-default-config4, leaderPayloadVersion=12] 07:01:59,838 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ShardManager - shard-manager-config4: Received role changed for member-2-shard-default-config4 from Candidate to Leader 07:01:59,838 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ShardManager - shard-manager-config4: All Shards are ready - data store config4 is ready 07:01:59,838 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:01:59,838 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config4 07:01:59,838 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-default-config4 07:01:59,838 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-default-config4 to akka://cluster-test/user/member-1-shard-default-config 07:01:59,838 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config4: updated peer member-1-shard-default-config4 on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-default-config4 on shard member-1-shard-default-config4 whose actor address is Actor[akka://cluster-test/user/member-1-shard-default-config#-1599491498] 07:01:59,838 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Creating Shard member-1-shard-astronauts-config4 07:01:59,838 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-astronauts-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-astronauts-config4 07:01:59,838 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-astronauts-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-astronauts-config4 to akka://cluster-test/user/member-1-shard-default-config 07:01:59,838 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config4: updated peer member-1-shard-astronauts-config4 on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-astronauts-config4 on shard member-1-shard-astronauts-config4 whose actor address is Actor[akka://cluster-test/user/member-1-shard-default-config#-1599491498] 07:01:59,838 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config4: Initializing shard [default] 07:01:59,838 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Shard member-1-shard-default-config4 is initialized 07:01:59,838 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config4: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config4, leaderId=member-2-shard-default-config4, leaderPayloadVersion=12] 07:01:59,838 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config4: Received role changed for member-1-shard-default-config4 from Candidate to Follower 07:02:00,167 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 07:02:00,203 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 07:02:00,207 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config4: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 07:02:00,207 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-default-config4 07:02:00,207 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-default-config4 to akka://cluster-test/user/member-2-shard-default-config4 07:02:00,207 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config4: All Shards are ready - data store config4 is ready 07:02:00,794 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 07:02:00,794 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config4: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 07:02:00,794 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config4 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-default-config4 07:02:00,794 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config4 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-default-config4 to akka://cluster-test/user/member-1-shard-default-config 07:02:00,794 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-astronauts-config4 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-astronauts-config4 07:02:00,794 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-astronauts-config4 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-astronauts-config4 to akka://cluster-test/user/member-1-shard-default-config 07:02:00,796 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config4: onRemoveShardReplica: RemoveShardReplica [shardName=default, memberName=MemberName{name=member-1}] 07:02:00,798 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config4: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 07:02:00,798 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config4: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-default-config4, primaryVersion=12] 07:02:00,798 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config4: Sending RemoveServer message to peer akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-default-config4 for shard member-1-shard-default-config4 07:02:00,812 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config4: Received RemoveServerReply{status=OK, leaderHint=member-2-shard-default-config4} for shard default 07:02:00,812 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config4: Leader shard successfully removed the replica shard default 07:02:00,836 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config4: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 07:02:00,836 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config4 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-default-config4 07:02:00,836 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config4 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-default-config4 to akka://cluster-test/user/member-2-shard-default-config4 07:02:00,836 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config4: All Shards are ready - data store config4 is ready 07:02:00,948 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-config4 07:02:00,949 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:02:00,949 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:02:00,975 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:02:00,975 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:02:00,984 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 07:02:00,998 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config4 07:02:01,0 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:02:01,0 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:02:02,24 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [01/22/2024 07:02:02.026] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:02:02,55 AM [test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$c#-552931548] to Actor[akka://test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 07:02:02,55 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config5 07:02:02,59 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config5 07:02:02,59 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config5: Creating local shard: member-1-shard-default-config5 07:02:02,68 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:02,68 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config5 07:02:02,68 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config5: Initializing shard [default] 07:02:02,68 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-default-config5 is initialized 07:02:02,219 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$c#-552931548] 07:02:02,230 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config5 07:02:02,235 AM [test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://test/user/$$c] to Actor[akka://test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 07:02:02,236 AM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config6 07:02:02,244 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config6 07:02:02,244 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config6: Creating local shard: member-1-shard-default-config6 07:02:02,244 AM [main] [INFO] ShardManager - shard-manager-config6: Received role changed for member-1-shard-default-config6 from null to Follower 07:02:02,255 AM [main] [INFO] ShardManager - shard-manager-config6: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://test 07:02:02,255 AM [main] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config6 with address akka://test/user/shardmanager-config6/member-2-shard-default-config6 07:02:02,255 AM [main] [INFO] ShardManager - shard-manager-config6: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config6, leaderId=member-2-shard-default-config6, leaderPayloadVersion=13] 07:02:02,255 AM [main] [INFO] ShardManager - shard-manager-config6: All Shards are ready - data store config6 is ready 07:02:02,357 AM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$c] 07:02:02,368 AM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config6 07:02:02,374 AM [test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$d#16831535] to Actor[akka://test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 07:02:02,374 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config7 07:02:02,377 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shard-manager-config7 07:02:02,385 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config7: onRemoveShardReplica: RemoveShardReplica [shardName=model-inventory, memberName=MemberName{name=member-1}] 07:02:02,385 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config7: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=model-inventory, waitUntilReady=true] 07:02:02,386 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config7: No shard found for model-inventory 07:02:02,386 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config7: Failed to find leader for shard model-inventory: received response: {} org.opendaylight.controller.cluster.datastore.exceptions.PrimaryNotFoundException: No primary shard found for model-inventory. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.findPrimary(ShardManager.java:1051) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:207) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManagerTest$TestShardManager.handleCommand(ShardManagerTest.java:2198) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 07:02:02,487 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$d#16831535] 07:02:02,498 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config7 07:02:02,501 AM [main] [INFO] ShardManagerTest - testAddShardReplicaWithPreExistingLocalReplicaLeader starting 07:02:02,506 AM [test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$e#-1035826882] to Actor[akka://test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 07:02:02,506 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config8 07:02:02,507 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shard-manager-config8 07:02:02,507 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config8: Creating local shard: member-1-shard-default-config8 07:02:02,516 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:02,516 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config8 07:02:02,516 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config8: Initializing shard [default] 07:02:02,517 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config8 is initialized 07:02:02,517 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config8: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config8, leaderId=member-1-shard-default-config8, leaderPayloadVersion=13] 07:02:02,517 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config8: Received role changed for member-1-shard-default-config8 from Candidate to Leader 07:02:02,517 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config8: All Shards are ready - data store config8 is ready 07:02:02,517 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config8: onAddShardReplica: AddShardReplica[ShardName=default] 07:02:02,520 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config8: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 07:02:02,520 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config8: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://test@127.0.0.1:2565/user/member-1-shard-default-config#718833929, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$dQrdkTAT@d5cbec0] 07:02:02,520 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config8: Local shard default already exists 07:02:02,520 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config8: findLocalShard : default 07:02:02,520 AM [main] [INFO] ShardManagerTest - testAddShardReplicaWithPreExistingLocalReplicaLeader ending 07:02:02,622 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$e#-1035826882] 07:02:02,632 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config8 07:02:02,635 AM [main] [INFO] ShardManagerTest - testOnReceiveFindLocalShardWaitForShardInitialized starting 07:02:02,641 AM [test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$f#548691757] to Actor[akka://test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 07:02:02,642 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config9 07:02:02,643 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shard-manager-config9 07:02:02,643 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config9: Creating local shard: member-1-shard-default-config9 07:02:02,652 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:02,652 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config9 07:02:02,652 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config9: findLocalShard : default 07:02:02,652 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config9: Scheduling 600 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config9, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 07:02:02,652 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config9: Initializing shard [default] 07:02:02,652 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-default-config9 is initialized 07:02:02,652 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-default-config9 is initialized - notifying 1 OnShardInitialized callbacks 07:02:02,657 AM [main] [INFO] ShardManagerTest - testOnReceiveFindLocalShardWaitForShardInitialized starting 07:02:02,758 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$f#548691757] 07:02:02,769 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config9 07:02:02,772 AM [main] [INFO] ShardManagerTest - testShardAvailabilityChangeOnMemberUnreachableAndLeadershipChange starting 07:02:02,781 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:02:02,795 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-3594050986210358965] 07:02:02,795 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 07:02:02,798 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 07:02:02,798 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 07:02:02,807 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 07:02:02,807 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 07:02:02,812 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 07:02:02,812 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 07:02:02,812 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 07:02:02,818 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config10] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 07:02:02,824 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config10 07:02:02,831 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-config10 07:02:02,831 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config10: Creating local shard: member-1-shard-default-config10 07:02:02,832 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config10: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 07:02:02,832 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config10 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config10/member-1-shard-default-config10 07:02:02,839 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 07:02:02,852 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-2233939326405781525] 07:02:02,853 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 07:02:02,856 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:02:02,856 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 07:02:02,863 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 07:02:02,863 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 07:02:02,875 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config10] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 07:02:02,878 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ShardManager - Starting ShardManager shard-manager-config10 07:02:02,878 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:02,878 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config10 07:02:02,878 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config10 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config10/member-1-shard-default-config10 07:02:02,878 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config10 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config10/member-1-shard-default-config10 to akka://cluster-test/user/member-1-shard-default-config 07:02:02,878 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config10: updated peer member-1-shard-default-config10 on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config10/member-1-shard-default-config10 on shard member-1-shard-default-config10 whose actor address is Actor[akka://cluster-test/user/member-1-shard-default-config#281568757] 07:02:02,878 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config10: Initializing shard [default] 07:02:02,878 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-default-config10 is initialized 07:02:02,878 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config10: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config10, leaderId=member-2-shard-default-config10, leaderPayloadVersion=13] 07:02:02,878 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config10: Received role changed for member-1-shard-default-config10 from Candidate to Follower 07:02:02,892 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config10 07:02:02,892 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config10: Creating local shard: member-2-shard-default-config10 07:02:02,893 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:02,893 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-2-shard-default-config10 07:02:02,893 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config10: Initializing shard [default] 07:02:02,893 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Shard member-2-shard-default-config10 is initialized 07:02:02,894 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config10: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-config10, leaderId=member-2-shard-default-config10, leaderPayloadVersion=13] 07:02:02,894 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config10: Received role changed for member-2-shard-default-config10 from Candidate to Leader 07:02:02,894 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config10: All Shards are ready - data store config10 is ready 07:02:02,921 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 07:02:02,926 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 07:02:02,927 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config10: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 07:02:02,927 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config10 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config10/member-1-shard-default-config10 07:02:02,927 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config10 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config10/member-1-shard-default-config10 to akka://cluster-test/user/member-2-shard-default-config 07:02:02,927 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config10: All Shards are ready - data store config10 is ready 07:02:03,821 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 07:02:03,821 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config10: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 07:02:03,821 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config10 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config10/member-2-shard-default-config10 07:02:03,821 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config10 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config10/member-2-shard-default-config10 to akka://cluster-test/user/member-1-shard-default-config 07:02:03,821 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config10: All Shards are ready - data store config10 is ready 07:02:03,821 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config10: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 07:02:03,822 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config10: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config10/member-2-shard-default-config10, primaryVersion=13] 07:02:03,822 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 07:02:03,823 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Marking Leader member-2-shard-default-config10 as unavailable. 07:02:03,823 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config10: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 07:02:03,823 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config10: Scheduling 1200 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config10, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=member-2-shard-default-config10, activeMember=true] 07:02:03,884 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config10: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 07:02:03,884 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config10 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config10/member-2-shard-default-config10 07:02:03,884 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config10 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config10/member-2-shard-default-config10 to akka://cluster-test/user/member-2-shard-default-config 07:02:03,884 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config10: All Shards are ready - data store config10 is ready 07:02:05,38 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config10: Received ShardNotInitializedTimeout message for shard default 07:02:05,38 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config10: Returning NoShardLeaderException for shard default 07:02:05,39 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config10: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config10, leaderId=member-1-shard-default-config10, leaderPayloadVersion=13] 07:02:05,39 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config10: All Shards are ready - data store config10 is ready 07:02:05,39 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config10: Received role changed for member-1-shard-default-config10 from Follower to Leader 07:02:05,39 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config10: All Shards are ready - data store config10 is ready 07:02:05,39 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config10: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 07:02:05,39 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config10: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/member-1-shard-default-config#281568757, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$dQrdkTAT@5a2266a3] 07:02:05,39 AM [main] [INFO] ShardManagerTest - testShardAvailabilityChangeOnMemberUnreachableAndLeadershipChange ending 07:02:05,141 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-config10 07:02:05,149 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:02:05,150 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:02:05,150 AM [cluster-test-akka.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:02:05,150 AM [cluster-test-akka.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:02:05,158 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 07:02:05,192 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-config10 07:02:05,193 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:02:05,193 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [01/22/2024 07:02:06.223] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:02:06,241 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config11 07:02:06,242 AM [test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://test/user/shardmanager-config11] to Actor[akka://test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 07:02:06,244 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config11 07:02:06,266 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:06,266 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config11: onAddShardReplica: AddShardReplica[ShardName=astronauts] 07:02:06,267 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config11: Sending AddServer message to peer akka://test@127.0.0.1:2565/system/testProbe-25#-63686119 for shard member-1-shard-astronauts-config11 07:02:06,278 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config11: onRemoveShardReplica: RemoveShardReplica [shardName=astronauts, memberName=MemberName{name=member-2}] 07:02:06,279 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config11: A shard replica operation for astronauts is already in progress 07:02:06,383 AM [main] [INFO] ShardManagerTest - testPerShardDatastoreContext starting 07:02:06,395 AM [test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$g#-966498103] to Actor[akka://test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 07:02:06,395 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config12 07:02:06,398 AM [test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config12 07:02:06,399 AM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config12: Creating local shard: member-1-shard-default-config12 07:02:06,399 AM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config12: Creating local shard: member-1-shard-topology-config12 07:02:06,406 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:06,406 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config12 07:02:06,406 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Creating Shard member-1-shard-topology-config12 07:02:06,408 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-default-config12 07:02:06,408 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-topology-config12 07:02:06,460 AM [main] [INFO] ShardManagerTest - testPerShardDatastoreContext ending 07:02:06,561 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/default1#-770574261] 07:02:06,572 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/topology2#1668404240] 07:02:06,573 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$g#-966498103] 07:02:06,573 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config12 07:02:06,575 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForLocalLeaderShard starting 07:02:06,578 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config13 07:02:06,580 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config13 07:02:06,580 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config13: Creating local shard: member-1-shard-default-config13 07:02:06,590 AM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:06,590 AM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config13 07:02:06,590 AM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config13: Initializing shard [default] 07:02:06,590 AM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-default-config13 is initialized 07:02:06,590 AM [test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config13: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config13, leaderId=member-1-shard-default-config13, leaderPayloadVersion=13] 07:02:06,590 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config13: Received role changed for member-1-shard-default-config13 from Candidate to Leader 07:02:06,590 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config13: All Shards are ready - data store config13 is ready 07:02:06,590 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config13: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 07:02:06,590 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config13: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://test@127.0.0.1:2565/user/member-1-shard-default-config#1568502176, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$dQrdkTAT@4b0f2bf] 07:02:06,591 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForLocalLeaderShard ending 07:02:06,692 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$h#-1333005571] 07:02:06,703 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config13 07:02:06,706 AM [main] [INFO] ShardManagerTest - testShardAvailabilityChangeOnMemberWithNameContainedInLeaderIdUnreachable starting 07:02:06,714 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:02:06,727 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2562] with UID [5654739431926336314] 07:02:06,728 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Starting up, Akka version [2.6.21] ... 07:02:06,730 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Registered cluster JMX MBean [akka:type=Cluster] 07:02:06,730 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Started up successfully 07:02:06,745 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 07:02:06,745 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 07:02:06,747 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config14 07:02:06,747 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config14] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 07:02:06,748 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Node [akka://cluster-test@127.0.0.1:2562] is JOINING itself (with roles [member-256, dc-default], version [0.0.0]) and forming new cluster 07:02:06,748 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - is the new leader among reachable nodes (more leaders may exist) 07:02:06,748 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Leader is moving node [akka://cluster-test@127.0.0.1:2562] to [Up] 07:02:06,765 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config14 07:02:06,765 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config14: Creating local shard: member-256-shard-default-config14 07:02:06,766 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config14: Received MemberUp: memberName: MemberName{name=member-256}, address: akka://cluster-test@127.0.0.1:2562 07:02:06,766 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-256-shard-default-config14 with address akka://cluster-test@127.0.0.1:2562/user/shardmanager-config14/member-256-shard-default-config14 07:02:06,770 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:02:06,787 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-5329280168524364836] 07:02:06,787 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 07:02:06,787 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:02:06,787 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 07:02:06,793 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 07:02:06,793 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 07:02:06,806 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config14 07:02:06,810 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config14] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 07:02:06,813 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:06,813 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-256-shard-default-config14 07:02:06,813 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-256-shard-default-config14 with address akka://cluster-test@127.0.0.1:2562/user/shardmanager-config14/member-256-shard-default-config14 07:02:06,813 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-256-shard-default-config14 with address akka://cluster-test@127.0.0.1:2562/user/shardmanager-config14/member-256-shard-default-config14 to akka://cluster-test/user/member-256-shard-default-config 07:02:06,813 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config14: updated peer member-256-shard-default-config14 on member MemberName{name=member-256} with address akka://cluster-test@127.0.0.1:2562/user/shardmanager-config14/member-256-shard-default-config14 on shard member-256-shard-default-config14 whose actor address is Actor[akka://cluster-test/user/member-256-shard-default-config#1394137433] 07:02:06,813 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config14: Initializing shard [default] 07:02:06,813 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Shard member-256-shard-default-config14 is initialized 07:02:06,813 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config14: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-256-shard-default-config14, leaderId=member-256-shard-default-config14, leaderPayloadVersion=13] 07:02:06,813 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config14: Received role changed for member-256-shard-default-config14 from Candidate to Leader 07:02:06,813 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config14: All Shards are ready - data store config14 is ready 07:02:06,816 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-config14 07:02:06,816 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config14: Creating local shard: member-2-shard-default-config14 07:02:06,817 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:06,817 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-2-shard-default-config14 07:02:06,817 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config14: Initializing shard [default] 07:02:06,817 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardInformation - Shard member-2-shard-default-config14 is initialized 07:02:06,817 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config14: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-config14, leaderId=member-256-shard-default-config14, leaderPayloadVersion=13] 07:02:06,817 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config14: Received role changed for member-2-shard-default-config14 from Candidate to Follower 07:02:06,873 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 07:02:06,876 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2562] 07:02:06,877 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config14: Received MemberUp: memberName: MemberName{name=member-256}, address: akka://cluster-test@127.0.0.1:2562 07:02:06,877 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-256-shard-default-config14 with address akka://cluster-test@127.0.0.1:2562/user/shardmanager-config14/member-256-shard-default-config14 07:02:06,877 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-256-shard-default-config14 with address akka://cluster-test@127.0.0.1:2562/user/shardmanager-config14/member-256-shard-default-config14 to akka://cluster-test/user/member-2-shard-default-config 07:02:06,877 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config14: All Shards are ready - data store config14 is ready 07:02:07,764 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 07:02:07,765 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config14: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 07:02:07,765 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config14 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config14/member-2-shard-default-config14 07:02:07,765 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config14 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config14/member-2-shard-default-config14 to akka://cluster-test/user/member-256-shard-default-config 07:02:07,765 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config14: All Shards are ready - data store config14 is ready 07:02:07,765 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config14: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 07:02:07,766 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config14: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2562/user/member-256-shard-default-config#1394137433, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$dQrdkTAT@58c74364] 07:02:07,768 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 07:02:07,768 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config14: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 07:02:07,768 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config14: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2562/user/member-256-shard-default-config#1394137433, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$dQrdkTAT@58c74364] 07:02:07,770 AM [main] [INFO] ShardManagerTest - testShardAvailabilityChangeOnMemberWithNameContainedInLeaderIdUnreachable ending 07:02:07,818 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config14: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 07:02:07,818 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config14 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config14/member-2-shard-default-config14 07:02:07,818 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config14 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config14/member-2-shard-default-config14 to akka://cluster-test/user/member-2-shard-default-config 07:02:07,818 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config14: All Shards are ready - data store config14 is ready 07:02:07,872 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config14 07:02:07,873 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:02:07,873 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [01/22/2024 07:02:07.883] [cluster-test-akka.remote.default-remote-dispatcher-6] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-92)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [01/22/2024 07:02:07.883] [cluster-test-akka.remote.default-remote-dispatcher-7] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-92)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:02:07,902 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config14 07:02:07,903 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:02:07,903 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:02:08,830 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - is the new leader among reachable nodes (more leaders may exist) 07:02:08,924 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [01/22/2024 07:02:08.926] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:02:08,946 AM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config15 07:02:08,947 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shard-manager-config15 07:02:08,947 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config15: Creating local shard: member-1-shard-default-config15 07:02:08,948 AM [main] [INFO] ShardManager - shard-manager-config15: Received role changed for member-1-shard-default-config15 from Follower to Leader 07:02:09,49 AM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$d] 07:02:09,60 AM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config15 07:02:09,65 AM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config16 07:02:09,73 AM [test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config16 07:02:09,73 AM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config16: Creating local shard: member-1-shard-default-config16 07:02:09,167 AM [main] [INFO] ShardManager - shard-manager-config16: Received role changed for member-1-shard-default-config16 from Candidate to Leader 07:02:09,167 AM [main] [INFO] ShardManager - shard-manager-config16: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config16, leaderId=member-1-shard-default-config16, leaderPayloadVersion=13] 07:02:09,167 AM [main] [INFO] ShardManager - shard-manager-config16: All Shards are ready - data store config16 is ready 07:02:09,269 AM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$e] 07:02:09,279 AM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config16 07:02:09,282 AM [main] [INFO] ShardManagerTest - testOnCreateShardWithLocalMemberNotInShardConfig starting 07:02:09,285 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config17 07:02:09,286 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shard-manager-config17 07:02:09,286 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:09,289 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config17: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=foo-ns, moduleName=foo-module, shardName=foo, shardMemberNames=[MemberName{name=member-5}, MemberName{name=member-6}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@6e736af1] 07:02:09,290 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config17 doCreateShard: shardId: member-1-shard-foo-config17, memberNames: [MemberName{name=member-5}, MemberName{name=member-6}], peerAddresses: {}, isActiveMember: false 07:02:09,291 AM [test-shard-dispatcher-15] [INFO] Shard - Shard created : member-1-shard-foo-config17, persistent : true 07:02:09,293 AM [test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-1-shard-foo-config17 with journal batch size 1 07:02:09,293 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config17: findLocalShard : foo 07:02:09,293 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config17: Scheduling 60000 ms timer to wait for shard ShardInformation [shardId=member-1-shard-foo-config17, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=false] 07:02:09,293 AM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$i/member-1-shard-foo-config17/shard-member-1:snapshot-read#1593777585] 07:02:09,294 AM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$i/member-1-shard-foo-config17/member-1-shard-foo-config17-notifier#52490682 created and ready for shard:member-1-shard-foo-config17 07:02:09,297 AM [test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-foo-config17: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 07:02:09,297 AM [test-shard-dispatcher-15] [INFO] Shard - member-1-shard-foo-config17: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:02:09,297 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config17: Initializing shard [foo] 07:02:09,297 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-foo-config17 is initialized 07:02:09,297 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-foo-config17 is initialized - notifying 1 OnShardInitialized callbacks 07:02:09,297 AM [test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-foo-config17: onLeadershipChange, isLeader: false, hasLeader : false 07:02:09,297 AM [test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-foo-config17: onStateChanged: Closing all transaction chains because shard member-1-shard-foo-config17 is no longer the leader 07:02:09,297 AM [test-shard-dispatcher-15] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 07:02:09,297 AM [test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config17 , received role change from null to Follower 07:02:09,297 AM [main] [INFO] ShardManagerTest - testOnCreateShardWithLocalMemberNotInShardConfig ending 07:02:09,297 AM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config17 , registered listener akka://test/user/$i 07:02:09,297 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config17: Received role changed for member-1-shard-foo-config17 from null to Follower 07:02:09,399 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$i#-1102400327] 07:02:09,411 AM [test-shard-dispatcher-15] [INFO] Shard - Stopping Shard member-1-shard-foo-config17 07:02:09,411 AM [test-shard-dispatcher-15] [DEBUG] MessageAssembler - member-1-shard-foo-config17 (Follower): Closing 07:02:09,411 AM [test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-foo-config17: Aborting 0 pending queued transactions 07:02:09,412 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config17 07:02:09,414 AM [main] [INFO] ShardManagerTest - testServerRemovedShardActorNotRunning starting 07:02:09,418 AM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-config18 07:02:09,419 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shard-manager-config18 07:02:09,419 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config18: Creating local shard: member-1-shard-default-config18 07:02:09,419 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config18: Creating local shard: member-1-shard-people-config18 07:02:09,528 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config18: findLocalShard : people 07:02:09,528 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config18: Returning NotInitializedException for shard people 07:02:09,528 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config18: findLocalShard : default 07:02:09,528 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config18: Returning NotInitializedException for shard default 07:02:09,530 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config18 : Local Shard replica for shard default has been removed 07:02:09,530 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config18: persisting the shard list [people] 07:02:09,530 AM [main] [INFO] ShardManagerTest - testServerRemovedShardActorNotRunning ending 07:02:09,531 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config18 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 07:02:09,531 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config18: Successfully deleted prior snapshots 07:02:09,632 AM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$f] 07:02:09,643 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config18 07:02:09,647 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config19 07:02:09,649 AM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-config19 07:02:09,649 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config19: Creating local shard: member-1-shard-default-config19 07:02:09,658 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:09,658 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config19 07:02:09,658 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config19: Initializing shard [default] 07:02:09,658 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Shard member-1-shard-default-config19 is initialized 07:02:09,658 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config19: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 07:02:09,658 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config19: Returning NoShardLeaderException for shard default 07:02:09,760 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$j#-1049402272] 07:02:09,771 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config19 07:02:09,773 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForFollowerShardWithNoInitialLeaderId starting 07:02:09,775 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config20 07:02:09,777 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shard-manager-config20 07:02:09,777 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config20: Creating local shard: member-1-shard-default-config20 07:02:09,786 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:09,786 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config20 07:02:09,786 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config20: Initializing shard [default] 07:02:09,786 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-default-config20 is initialized 07:02:09,786 AM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - shard-manager-config20: Received role changed for member-1-shard-default-config20 from Candidate to Follower 07:02:09,786 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config20: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 07:02:09,786 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config20: Returning NoShardLeaderException for shard default 07:02:09,787 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config20: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config20, leaderId=member-1-shard-default-config20, leaderPayloadVersion=13] 07:02:09,787 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config20: All Shards are ready - data store config20 is ready 07:02:09,787 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config20: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 07:02:09,787 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config20: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://test@127.0.0.1:2565/user/member-1-shard-default-config#2003014111, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$dQrdkTAT@24acd393] 07:02:09,787 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForFollowerShardWithNoInitialLeaderId starting 07:02:09,888 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$k#1887066079] 07:02:09,899 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config20 07:02:09,904 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config21 07:02:09,906 AM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-config21 07:02:09,925 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:09,925 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config21: onAddShardReplica: AddShardReplica[ShardName=astronauts] 07:02:09,925 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config21: Sending AddServer message to peer akka://test@127.0.0.1:2565/system/testProbe-43#-1905055614 for shard member-1-shard-astronauts-config21 07:02:09,936 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config21: onAddShardReplica: AddShardReplica[ShardName=astronauts] 07:02:09,937 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config21: A shard replica operation for astronauts is already in progress 07:02:10,41 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithCandidateShard starting 07:02:10,43 AM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-config22 07:02:10,44 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shard-manager-config22 07:02:10,44 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config22: Creating local shard: member-1-shard-default-config22 07:02:10,53 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:10,53 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config22 07:02:10,53 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config22: Initializing shard [default] 07:02:10,53 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-default-config22 is initialized 07:02:10,53 AM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - shard-manager-config22: Received role changed for member-1-shard-default-config22 from null to Candidate 07:02:10,53 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config22: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 07:02:10,53 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config22: Scheduling 1200 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config22, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Candidate, leaderId=null, activeMember=true] 07:02:11,266 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config22: Received ShardNotInitializedTimeout message for shard default 07:02:11,266 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config22: Returning NoShardLeaderException for shard default 07:02:11,266 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithCandidateShard ending 07:02:11,368 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$l#-597722640] 07:02:11,379 AM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-config22 07:02:11,381 AM [main] [INFO] ShardManagerTest - testAddShardReplicaWithAddServerReplyFailure starting 07:02:11,384 AM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-config23 07:02:11,386 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shard-manager-config23 07:02:11,405 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:11,416 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config23: onAddShardReplica: AddShardReplica[ShardName=astronauts] 07:02:11,417 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config23: Sending AddServer message to peer akka://test@127.0.0.1:2565/system/testProbe-48#-628506162 for shard member-1-shard-astronauts-config23 07:02:11,418 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config23: Received AddServerReply{status=TIMEOUT} for shard astronauts from leader akka://test@127.0.0.1:2565/system/testProbe-48#-628506162 07:02:11,418 AM [test-akka.actor.default-dispatcher-5] [WARN] ShardManager - shard-manager-config23: Leader failed to add shard replica astronauts with status TIMEOUT 07:02:11,418 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config23: findLocalShard : astronauts 07:02:11,418 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config23: Local shard astronauts not found - shards present: [] 07:02:11,419 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config23: onAddShardReplica: AddShardReplica[ShardName=astronauts] 07:02:11,419 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config23: Sending AddServer message to peer akka://test@127.0.0.1:2565/system/testProbe-48#-628506162 for shard member-1-shard-astronauts-config23 07:02:11,420 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config23: Received AddServerReply{status=NO_LEADER} for shard astronauts from leader akka://test@127.0.0.1:2565/system/testProbe-48#-628506162 07:02:11,420 AM [test-akka.actor.default-dispatcher-16] [WARN] ShardManager - shard-manager-config23: Leader failed to add shard replica astronauts with status NO_LEADER 07:02:11,420 AM [main] [INFO] ShardManagerTest - testAddShardReplicaWithAddServerReplyFailure ending 07:02:11,521 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-astronauts-config#-820974184] 07:02:11,532 AM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/shardmanager-config23] 07:02:11,532 AM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-config23 07:02:11,537 AM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config24 07:02:11,539 AM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-config24 07:02:11,539 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config24: Creating local shard: member-1-shard-default-config24 07:02:11,539 AM [main] [INFO] ShardManager - shard-manager-config24: Received role changed for member-1-shard-default-config24 from Candidate to Follower 07:02:11,539 AM [main] [INFO] ShardManager - shard-manager-config24 Received follower initial sync status for member-1-shard-default-config24 status sync done true 07:02:11,539 AM [main] [INFO] ShardManager - shard-manager-config24 Received follower initial sync status for member-1-shard-default-config24 status sync done false 07:02:11,641 AM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$g] 07:02:11,651 AM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config24 07:02:11,655 AM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config25 07:02:11,656 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config25 07:02:11,656 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config25: Creating local shard: member-1-shard-default-config25 07:02:11,657 AM [main] [INFO] ShardManager - shard-manager-config25: Received role changed for unknown from Candidate to Leader 07:02:11,758 AM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$h] 07:02:11,769 AM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config25 07:02:11,771 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithUninitializedShard starting 07:02:11,773 AM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-config26 07:02:11,774 AM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-config26 07:02:11,774 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config26: Creating local shard: member-1-shard-default-config26 07:02:11,785 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:11,785 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config26 07:02:11,785 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config26: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 07:02:11,785 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config26: Scheduling 600 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config26, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 07:02:12,396 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config26: Received ShardNotInitializedTimeout message for shard default 07:02:12,396 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config26: Returning NotInitializedException for shard default 07:02:12,397 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config26: Initializing shard [default] 07:02:12,397 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Shard member-1-shard-default-config26 is initialized 07:02:12,597 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithUninitializedShard ending 07:02:12,698 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$m#-1519696168] 07:02:12,709 AM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-config26 07:02:12,716 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config27 07:02:12,717 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shard-manager-config27 07:02:12,717 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config27: Creating local shard: member-1-shard-default-config27 07:02:12,727 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:12,727 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config27 07:02:12,727 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config27: Initializing shard [default] 07:02:12,727 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Shard member-1-shard-default-config27 is initialized 07:02:12,727 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config27: findLocalShard : default 07:02:12,828 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$n#886790683] 07:02:12,839 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config27 07:02:12,842 AM [main] [INFO] ShardManagerTest - testRegisterForShardLeaderChanges starting 07:02:12,844 AM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-config28 07:02:12,845 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shard-manager-config28 07:02:12,845 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config28: Creating local shard: member-1-shard-default-config28 07:02:12,855 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:12,855 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config28 07:02:12,855 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config28: Initializing shard [default] 07:02:12,855 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-default-config28 is initialized 07:02:12,936 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config28: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@fe5d47e 07:02:12,937 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config28: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config28, leaderId=member-1-shard-default-config28, leaderPayloadVersion=13] 07:02:12,967 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config28: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config28, leaderId=member-1-shard-default-config28, leaderPayloadVersion=13] 07:02:13,467 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config28: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config28, leaderId=null, leaderPayloadVersion=13] 07:02:13,479 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config28: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config28, leaderId=member-2-shard-default-config28, leaderPayloadVersion=13] 07:02:13,490 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config28: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config28, leaderId=member-1-shard-default-config28, leaderPayloadVersion=13] 07:02:13,990 AM [main] [INFO] ShardManagerTest - testRegisterForShardLeaderChanges ending 07:02:14,91 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$o#567068673] 07:02:14,102 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config28 07:02:14,107 AM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config29 07:02:14,108 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shard-manager-config29 07:02:14,108 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config29: Creating local shard: member-1-shard-default-config29 07:02:14,209 AM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$i] 07:02:14,220 AM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config29 07:02:14,224 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config30 07:02:14,225 AM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-config30 07:02:14,225 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config30: Creating local shard: member-1-shard-default-config30 07:02:14,234 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:14,234 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config30 07:02:14,234 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config30: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=non-existent, waitUntilReady=false] 07:02:14,234 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config30: No shard found for non-existent 07:02:14,336 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$p#-873668309] 07:02:14,347 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config30 07:02:14,352 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config31 07:02:14,352 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config31 07:02:14,362 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config31: onAddShardReplica: AddShardReplica[ShardName=model-inventory] 07:02:14,362 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config31: No module configuration exists for shard model-inventory 07:02:14,464 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$q#275482306] 07:02:14,475 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config31 07:02:14,476 AM [main] [INFO] ShardManagerTest - testAddShardReplicaWithPreExistingReplicaInRemoteShardLeader starting 07:02:14,478 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config32 07:02:14,479 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shard-manager-config32 07:02:14,479 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config32: Creating local shard: member-1-shard-default-config32 07:02:14,589 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:14,589 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config32 07:02:14,589 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config32: Initializing shard [default] 07:02:14,589 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config32 is initialized 07:02:14,589 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config32: Received MemberUp: memberName: MemberName{name=leader-member}, address: akka://test 07:02:14,589 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer leader-member-shard-default-config32 with address akka://test/user/shardmanager-config32/leader-member-shard-default-config32 07:02:14,589 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer leader-member-shard-default-config32 with address akka://test/user/shardmanager-config32/leader-member-shard-default-config32 to akka://test/user/member-1-shard-default-config 07:02:14,589 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config32: Received role changed for member-1-shard-default-config32 from Candidate to Follower 07:02:14,589 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config32: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config32, leaderId=leader-member-shard-default-config32, leaderPayloadVersion=13] 07:02:14,590 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config32: All Shards are ready - data store config32 is ready 07:02:14,590 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config32: onAddShardReplica: AddShardReplica[ShardName=default] 07:02:14,590 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config32: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 07:02:14,590 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config32: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://test/user/shardmanager-config32/leader-member-shard-default-config32, primaryVersion=13] 07:02:14,590 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config32: Sending AddServer message to peer akka://test/user/shardmanager-config32/leader-member-shard-default-config32 for shard member-1-shard-default-config32 07:02:14,590 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config32: Received AddServerReply{status=ALREADY_EXISTS} for shard default from leader akka://test/user/shardmanager-config32/leader-member-shard-default-config32 07:02:14,590 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config32: Local shard default already exists 07:02:14,641 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config32: findLocalShard : default 07:02:14,641 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config32: onAddShardReplica: AddShardReplica[ShardName=default] 07:02:14,641 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config32: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 07:02:14,641 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config32: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://test/user/shardmanager-config32/leader-member-shard-default-config32, primaryVersion=13] 07:02:14,642 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config32: Sending AddServer message to peer akka://test/user/shardmanager-config32/leader-member-shard-default-config32 for shard member-1-shard-default-config32 07:02:14,642 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config32: Received AddServerReply{status=ALREADY_EXISTS} for shard default from leader akka://test/user/shardmanager-config32/leader-member-shard-default-config32 07:02:14,642 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config32: Local shard default already exists 07:02:14,643 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-default-config32 07:02:14,643 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config32: onAddShardReplica: AddShardReplica[ShardName=default] 07:02:14,643 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config32: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 07:02:14,643 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config32: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://test/user/shardmanager-config32/leader-member-shard-default-config32, primaryVersion=13] 07:02:14,643 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config32: Sending AddServer message to peer akka://test/user/shardmanager-config32/leader-member-shard-default-config32 for shard member-1-shard-default-config32 07:02:14,756 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config32: AddServer request to akka://test/user/shardmanager-config32/leader-member-shard-default-config32 for default failed akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://test/), Path(/user/shardmanager-config32/leader-member-shard-default-config32)]] after [100 ms]. Message of type [org.opendaylight.controller.cluster.raft.messages.AddServer]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 07:02:14,756 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config32: findLocalShard : default 07:02:14,757 AM [main] [INFO] ShardManagerTest - testAddShardReplicaWithPreExistingReplicaInRemoteShardLeader ending 07:02:14,858 AM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/shardmanager-config32] 07:02:14,869 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config32 07:02:14,871 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForRemoteShard starting 07:02:14,880 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 07:02:14,891 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-6099351074899376265] 07:02:14,891 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 07:02:14,893 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 07:02:14,893 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 07:02:14,897 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 07:02:14,897 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 07:02:14,906 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 07:02:14,906 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 07:02:14,907 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 07:02:14,910 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Starting ShardManager shard-manager-config33 07:02:14,910 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config33] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 07:02:14,915 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:02:14,917 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config33 07:02:14,917 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config33: Creating local shard: member-1-shard-default-config33 07:02:14,918 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config33: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 07:02:14,918 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-default-config33 07:02:14,926 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-3951747903172319797] 07:02:14,926 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 07:02:14,929 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:02:14,929 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 07:02:14,938 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 07:02:14,938 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 07:02:14,947 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config33] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 07:02:14,948 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config33 07:02:14,954 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config33 07:02:14,954 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config33: Creating local shard: member-2-shard-default-config33 07:02:14,954 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config33: Creating local shard: member-2-shard-astronauts-config33 07:02:14,955 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:14,955 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-2-shard-default-config33 07:02:14,955 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-2-shard-astronauts-config33 07:02:14,955 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config33: Initializing shard [astronauts] 07:02:14,955 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-2-shard-astronauts-config33 is initialized 07:02:14,955 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config33: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-astronauts-config33, leaderId=member-2-shard-astronauts-config33, leaderPayloadVersion=12] 07:02:14,955 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:14,955 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config33: Received role changed for member-2-shard-astronauts-config33 from Candidate to Leader 07:02:14,955 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config33 07:02:14,955 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-default-config33 07:02:14,955 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-default-config33 to akka://test/user/member-1-shard-default-config 07:02:14,955 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config33: updated peer member-1-shard-default-config33 on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-default-config33 on shard member-1-shard-default-config33 whose actor address is Actor[akka://test/user/member-1-shard-default-config#-675894643] 07:02:14,999 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 07:02:15,3 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config33: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 07:02:15,3 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-default-config33 07:02:15,3 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-default-config33 to akka://cluster-test/user/member-2-shard-astronauts-config 07:02:15,3 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 07:02:15,3 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-astronauts-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-astronauts-config33 07:02:15,3 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-astronauts-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-astronauts-config33 to akka://cluster-test/user/member-2-shard-astronauts-config 07:02:15,909 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 07:02:15,909 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config33: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 07:02:15,909 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config33 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33/member-2-shard-default-config33 07:02:15,909 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config33 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33/member-2-shard-default-config33 to akka://test/user/member-1-shard-default-config 07:02:15,910 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config33: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=astronauts, waitUntilReady=false] 07:02:15,910 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config33: findPrimary for astronauts forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33, visitedAddresses: [akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33] 07:02:15,917 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config33: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=astronauts, waitUntilReady=false] 07:02:15,917 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config33: Found primary for astronauts: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/member-2-shard-astronauts-config#64446929, primaryVersion=12] 07:02:15,925 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForRemoteShard ending 07:02:15,958 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config33: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 07:02:15,958 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config33 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33/member-2-shard-default-config33 07:02:15,958 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config33 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33/member-2-shard-default-config33 to akka://cluster-test/user/member-2-shard-astronauts-config 07:02:15,958 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-astronauts-config33 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33/member-2-shard-astronauts-config33 07:02:15,958 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-astronauts-config33 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33/member-2-shard-astronauts-config33 to akka://cluster-test/user/member-2-shard-astronauts-config 07:02:16,27 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config33 07:02:16,28 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:02:16,28 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:02:16,34 AM [cluster-test-akka.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:02:16,35 AM [cluster-test-akka.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:02:16,37 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 07:02:16,49 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config33 07:02:16,50 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:02:16,50 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:02:17,72 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [01/22/2024 07:02:17.074] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:02:17,97 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config34 07:02:17,100 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shard-manager-config34 07:02:17,100 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config34: Creating local shard: member-1-shard-default-config34 07:02:17,108 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config34: findLocalShard : default 07:02:17,108 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config34: Returning NotInitializedException for shard default 07:02:17,209 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$r#149186022] 07:02:17,220 AM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-config34 07:02:17,229 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config35 07:02:17,231 AM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-config35 07:02:17,231 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config35: Creating local shard: member-1-shard-default-config35 07:02:17,237 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:17,238 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config35 07:02:17,238 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config35: Initializing shard [default] 07:02:17,238 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-default-config35 is initialized 07:02:17,238 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config35: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config35, leaderId=member-1-shard-default-config35, leaderPayloadVersion=13] 07:02:17,238 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config35: Received role changed for member-1-shard-default-config35 from Candidate to Leader 07:02:17,238 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config35: All Shards are ready - data store config35 is ready 07:02:17,238 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config35: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=default, meberVotingStatusMap={member-2=true}] 07:02:17,238 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config35: findLocalShard : default 07:02:17,238 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config35: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-default-config35=true}, serversVisited=[]] to local shard akka://test/user/member-1-shard-default-config35 07:02:17,239 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config35: Received ServerChangeReply{status=OK} from local shard akka://test/user/member-1-shard-default-config35 07:02:17,239 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config35: ChangeServersVotingStatus succeeded for shard default 07:02:17,390 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-default-config35#-1679981498] 07:02:17,402 AM [main] [INFO] ShardManagerTest - testShutDown starting 07:02:17,405 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config36 07:02:17,407 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config36 07:02:17,407 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config36: Creating local shard: member-1-shard-shard2-config36 07:02:17,407 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config36: Creating local shard: member-1-shard-shard1-config36 07:02:17,416 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:17,416 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-shard2-config36 07:02:17,416 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-shard1-config36 07:02:17,416 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config36: Initializing shard [shard1] 07:02:17,416 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-shard1-config36 is initialized 07:02:17,416 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config36: Initializing shard [shard2] 07:02:17,416 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-shard2-config36 is initialized 07:02:17,416 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config36: Issuing gracefulStop to shard member-1-shard-shard2-config36 07:02:17,417 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config36: Issuing gracefulStop to shard member-1-shard-shard1-config36 07:02:17,417 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Shutting down ShardManager shard-manager-config36 - waiting on 2 shards 07:02:17,968 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-shard1-config36#-1333462568] 07:02:17,969 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-shard2-config36#-235190926] 07:02:17,969 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config36: All shards shutdown - sending PoisonPill to self 07:02:17,970 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config36 07:02:17,970 AM [main] [INFO] ShardManagerTest - testShutDown ending 07:02:18,73 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$t#-1692732144] 07:02:18,85 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForNonLocalLeaderShardBeforeMemberUp starting 07:02:18,88 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config37 07:02:18,89 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config37 07:02:18,89 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config37: Creating local shard: member-1-shard-default-config37 07:02:18,98 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:18,98 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config37 07:02:18,98 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config37: Initializing shard [default] 07:02:18,98 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-default-config37 is initialized 07:02:18,98 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config37: Received role changed for member-1-shard-default-config37 from Candidate to Follower 07:02:18,99 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManagerTest$TestShardManager - Received unhandled message LeaderStateChanged [memberId=member-1-shard-default-config37, leaderId=member-2-shard-default-config37, leaderPayloadVersion=13] 07:02:18,100 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config37: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 07:02:18,100 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config37: Returning NoShardLeaderException for shard default 07:02:18,100 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForNonLocalLeaderShardBeforeMemberUp ending 07:02:18,201 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$u#-1756394181] 07:02:18,212 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config37 07:02:18,214 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithNoRoleShard starting 07:02:18,216 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config38 07:02:18,217 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config38 07:02:18,217 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config38: Creating local shard: member-1-shard-default-config38 07:02:18,227 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:18,227 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config38 07:02:18,227 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config38: Initializing shard [default] 07:02:18,227 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Shard member-1-shard-default-config38 is initialized 07:02:18,227 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config38: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 07:02:18,228 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config38: Scheduling 1200 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config38, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 07:02:19,446 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config38: Received ShardNotInitializedTimeout message for shard default 07:02:19,446 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config38: Returning NoShardLeaderException for shard default 07:02:19,447 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithNoRoleShard ending 07:02:19,548 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$v#-533499035] 07:02:19,559 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config38 07:02:19,560 AM [main] [INFO] ShardManagerTest - testShardPersistenceWithRestoredData starting 07:02:19,563 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config39 07:02:19,564 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config39: onSnapshotOffer: ShardManagerSnapshot [ShardList = [default, astronauts] ] 07:02:19,564 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config39: adding shard astronauts 07:02:19,564 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config39: removing shard people 07:02:19,564 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config39 07:02:19,564 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config39: Creating local shard: member-1-shard-default-config39 07:02:19,565 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config39: Creating local shard: member-1-shard-astronauts-config39 07:02:19,673 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config39: findLocalShard : people 07:02:19,673 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config39: Local shard people not found - shards present: [default, astronauts] 07:02:19,673 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config39: findLocalShard : default 07:02:19,673 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config39: Returning NotInitializedException for shard default 07:02:19,673 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config39: findLocalShard : astronauts 07:02:19,673 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config39: Returning NotInitializedException for shard astronauts 07:02:19,674 AM [main] [INFO] ShardManagerTest - testShardPersistenceWithRestoredData ending 07:02:19,775 AM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$j] 07:02:19,786 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config39 07:02:19,788 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForNonLocalLeaderShard starting 07:02:19,791 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config40 07:02:19,792 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config40 07:02:19,792 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config40: Creating local shard: member-1-shard-default-config40 07:02:19,802 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:19,802 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config40 07:02:19,802 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config40: Initializing shard [default] 07:02:19,802 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-default-config40 is initialized 07:02:19,802 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config40: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://test 07:02:19,802 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config40 with address akka://test/user/shardmanager-config40/member-2-shard-default-config40 07:02:19,802 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config40 with address akka://test/user/shardmanager-config40/member-2-shard-default-config40 to akka://test/user/member-1-shard-default-config 07:02:19,802 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config40: Received role changed for member-1-shard-default-config40 from Candidate to Follower 07:02:19,802 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config40: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config40, leaderId=member-2-shard-default-config40, leaderPayloadVersion=12] 07:02:19,802 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config40: All Shards are ready - data store config40 is ready 07:02:19,802 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config40: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 07:02:19,802 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config40: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://test/user/shardmanager-config40/member-2-shard-default-config40, primaryVersion=12] 07:02:19,803 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForNonLocalLeaderShard ending 07:02:19,904 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$w#633380244] 07:02:19,915 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config40 07:02:19,917 AM [main] [INFO] ShardManagerTest - testShardAvailabilityOnChangeOfMemberReachability starting 07:02:19,926 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:02:19,943 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [2219645792787058589] 07:02:19,944 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 07:02:19,948 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 07:02:19,948 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 07:02:19,949 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 07:02:19,964 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 07:02:19,964 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config41] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 07:02:19,967 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 07:02:19,967 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 07:02:19,967 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 07:02:19,973 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config41 07:02:19,988 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config41 07:02:19,988 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config41: Creating local shard: member-1-shard-default-config41 07:02:19,989 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config41: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 07:02:19,989 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-1-shard-default-config41 07:02:19,989 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:02:20,3 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-7253546482668356487] 07:02:20,4 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 07:02:20,6 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:02:20,6 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 07:02:20,17 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 07:02:20,17 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 07:02:20,26 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config41] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 07:02:20,29 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config41 07:02:20,35 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:20,35 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config41 07:02:20,35 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-1-shard-default-config41 07:02:20,35 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-1-shard-default-config41 to akka://cluster-test/user/member-1-shard-default-config 07:02:20,35 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config41: updated peer member-1-shard-default-config41 on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-1-shard-default-config41 on shard member-1-shard-default-config41 whose actor address is Actor[akka://cluster-test/user/member-1-shard-default-config#-2065027362] 07:02:20,35 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config41: Initializing shard [default] 07:02:20,35 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-default-config41 is initialized 07:02:20,35 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config41: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config41, leaderId=member-2-shard-default-config41, leaderPayloadVersion=13] 07:02:20,35 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config41: Received role changed for member-1-shard-default-config41 from Candidate to Follower 07:02:20,54 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config41 07:02:20,54 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config41: Creating local shard: member-2-shard-default-config41 07:02:20,55 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:20,55 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-default-config41 07:02:20,55 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config41: Initializing shard [default] 07:02:20,55 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-default-config41 is initialized 07:02:20,55 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config41: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-config41, leaderId=member-2-shard-default-config41, leaderPayloadVersion=13] 07:02:20,55 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config41: Received role changed for member-2-shard-default-config41 from Candidate to Leader 07:02:20,55 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config41: All Shards are ready - data store config41 is ready 07:02:20,113 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 07:02:20,118 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 07:02:20,118 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config41: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 07:02:20,118 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-1-shard-default-config41 07:02:20,118 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-1-shard-default-config41 to akka://cluster-test/user/member-2-shard-default-config 07:02:20,118 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config41: All Shards are ready - data store config41 is ready 07:02:20,962 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 07:02:20,963 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config41: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 07:02:20,963 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config41/member-2-shard-default-config41 07:02:20,963 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config41/member-2-shard-default-config41 to akka://cluster-test/user/member-1-shard-default-config 07:02:20,963 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config41: All Shards are ready - data store config41 is ready 07:02:20,963 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config41: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 07:02:20,963 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config41: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config41/member-2-shard-default-config41, primaryVersion=13] 07:02:20,963 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 07:02:20,963 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Marking Leader member-2-shard-default-config41 as unavailable. 07:02:20,964 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config41: Received MemberRemoved: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 07:02:20,964 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config41: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 07:02:20,964 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config41: Scheduling 1200 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config41, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=member-2-shard-default-config41, activeMember=true] 07:02:21,45 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config41: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 07:02:21,45 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config41/member-2-shard-default-config41 07:02:21,45 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config41/member-2-shard-default-config41 to akka://cluster-test/user/member-2-shard-default-config 07:02:21,45 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config41: All Shards are ready - data store config41 is ready 07:02:22,181 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config41: Received ShardNotInitializedTimeout message for shard default 07:02:22,181 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config41: Returning NoShardLeaderException for shard default 07:02:22,182 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Received ReachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 07:02:22,182 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41 07:02:22,182 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41 to akka://cluster-test/user/member-1-shard-default-config 07:02:22,183 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Marking Leader member-2-shard-default-config41 as available. 07:02:22,183 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config41: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 07:02:22,183 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config41: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41, primaryVersion=13] 07:02:22,183 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config41: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 07:02:22,183 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41 07:02:22,183 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41 to akka://cluster-test/user/member-1-shard-default-config 07:02:22,183 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config41: All Shards are ready - data store config41 is ready 07:02:22,183 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 07:02:22,183 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Marking Leader member-2-shard-default-config41 as unavailable. 07:02:22,183 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config41: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 07:02:22,184 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config41: Scheduling 1200 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config41, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=member-2-shard-default-config41, activeMember=true] 07:02:22,184 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Received ReachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 07:02:22,184 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41 07:02:22,184 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41 to akka://cluster-test/user/member-1-shard-default-config 07:02:22,184 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-default-config41 is initialized - notifying 1 OnShardInitialized callbacks 07:02:22,184 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Marking Leader member-2-shard-default-config41 as available. 07:02:22,184 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-default-config41 is ready - notifying 1 OnShardInitialized callbacks 07:02:22,184 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config41: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41, primaryVersion=13] 07:02:22,184 AM [main] [INFO] ShardManagerTest - testShardAvailabilityOnChangeOfMemberReachability ending 07:02:22,286 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config41 07:02:22,287 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:02:22,287 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:02:22,299 AM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:02:22,300 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:02:22,303 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 07:02:22,326 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-config41 07:02:22,327 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:02:22,327 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:02:23,352 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [01/22/2024 07:02:23.354] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:02:23,371 AM [main] [INFO] ShardManagerTest - testAddShardReplica starting 07:02:23,380 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 07:02:23,389 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [4796789552930765291] 07:02:23,389 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 07:02:23,392 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 07:02:23,392 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 07:02:23,403 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 07:02:23,403 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 07:02:23,407 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shardmanager-config42 07:02:23,407 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config42] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 07:02:23,407 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 07:02:23,407 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 07:02:23,407 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 07:02:23,413 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 07:02:23,415 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Recovery complete : shardmanager-config42 07:02:23,416 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shardmanager-config42: Creating local shard: member-1-shard-default-config42 07:02:23,416 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shardmanager-config42: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 07:02:23,416 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-default-config42 07:02:23,423 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-1802031151773110066] 07:02:23,425 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 07:02:23,425 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:02:23,425 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 07:02:23,426 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 07:02:23,434 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 07:02:23,443 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config42] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 07:02:23,445 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Starting ShardManager shardmanager-config42 07:02:23,447 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:23,447 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config42 07:02:23,448 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-default-config42 07:02:23,448 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-default-config42 to akka://cluster-test/user/member-1-shard-default-config 07:02:23,448 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shardmanager-config42: updated peer member-1-shard-default-config42 on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-default-config42 on shard member-1-shard-default-config42 whose actor address is Actor[akka://cluster-test/user/member-1-shard-default-config#1646369222] 07:02:23,456 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shardmanager-config42 07:02:23,456 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shardmanager-config42: Creating local shard: member-2-shard-default-config42 07:02:23,457 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shardmanager-config42: Creating local shard: member-2-shard-astronauts-config42 07:02:23,457 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:23,457 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-default-config42 07:02:23,457 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-astronauts-config42 07:02:23,457 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shardmanager-config42: Initializing shard [astronauts] 07:02:23,457 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-astronauts-config42 is initialized 07:02:23,457 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shardmanager-config42: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-astronauts-config42, leaderId=member-2-shard-astronauts-config42, leaderPayloadVersion=12] 07:02:23,457 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shardmanager-config42: Received role changed for member-2-shard-astronauts-config42 from Candidate to Leader 07:02:23,491 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 07:02:23,498 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 07:02:23,499 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shardmanager-config42: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 07:02:23,499 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-default-config42 07:02:23,499 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-default-config42 to akka://cluster-test/user/member-2-shard-astronauts-config 07:02:23,499 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-astronauts-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-astronauts-config42 07:02:23,499 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-astronauts-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-astronauts-config42 to akka://cluster-test/user/member-2-shard-astronauts-config 07:02:24,416 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 07:02:24,417 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shardmanager-config42: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 07:02:24,417 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config42 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42/member-2-shard-default-config42 07:02:24,417 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config42 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42/member-2-shard-default-config42 to akka://cluster-test/user/member-1-shard-default-config 07:02:24,424 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shardmanager-config42: onAddShardReplica: AddShardReplica[ShardName=astronauts] 07:02:24,424 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shardmanager-config42: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=astronauts, waitUntilReady=true] 07:02:24,424 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shardmanager-config42: findPrimary for astronauts forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42, visitedAddresses: [akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42] 07:02:24,427 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shardmanager-config42: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=astronauts, waitUntilReady=true] 07:02:24,427 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shardmanager-config42: Found primary for astronauts: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/member-2-shard-astronauts-config, primaryVersion=12] 07:02:24,430 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shardmanager-config42: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2559/user/member-2-shard-astronauts-config for shard member-1-shard-astronauts-config42 07:02:24,444 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shardmanager-config42: Received AddServerReply{status=OK, leaderHint=member-2-shard-astronauts-config42} for shard astronauts from leader akka://cluster-test@127.0.0.1:2559/user/member-2-shard-astronauts-config 07:02:24,444 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shardmanager-config42: Leader shard successfully added the replica shard astronauts 07:02:24,444 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-astronauts-config42 07:02:24,444 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shardmanager-config42: persisting the shard list [default, astronauts] 07:02:24,447 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shardmanager-config42: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 07:02:24,447 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config42 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42/member-2-shard-default-config42 07:02:24,447 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config42 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42/member-2-shard-default-config42 to akka://cluster-test/user/member-2-shard-astronauts-config 07:02:24,447 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-astronauts-config42 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42/member-2-shard-astronauts-config42 07:02:24,447 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-astronauts-config42 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42/member-2-shard-astronauts-config42 to akka://cluster-test/user/member-2-shard-astronauts-config 07:02:24,447 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shardmanager-config42 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 07:02:24,449 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shardmanager-config42: Successfully deleted prior snapshots 07:02:24,473 AM [main] [INFO] ShardManagerTest - testAddShardReplica ending 07:02:24,575 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shardmanager-config42 07:02:24,576 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:02:24,576 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:02:24,583 AM [cluster-test-akka.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:02:24,583 AM [cluster-test-akka.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 07:02:24,596 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shardmanager-config42 07:02:24,597 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:02:24,597 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:02:25,619 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [01/22/2024 07:02:25.622] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 07:02:25,643 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config43 07:02:25,646 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config43 07:02:25,646 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config43: Creating local shard: member-1-shard-default-config43 07:02:25,652 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:25,652 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config43 07:02:25,652 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config43: Initializing shard [default] 07:02:25,652 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config43 is initialized 07:02:25,653 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config43: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config43, leaderId=member-1-shard-default-config43, leaderPayloadVersion=13] 07:02:25,653 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config43: Received role changed for member-1-shard-default-config43 from Candidate to Leader 07:02:25,653 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config43: All Shards are ready - data store config43 is ready 07:02:25,653 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config43: onRemoveShardReplica: RemoveShardReplica [shardName=default, memberName=MemberName{name=member-1}] 07:02:25,653 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config43: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 07:02:25,653 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config43: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://test@127.0.0.1:2565/user/member-1-shard-default-config43#610970420, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$dQrdkTAT@4919a94a] 07:02:25,653 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config43: Sending RemoveServer message to peer akka://test@127.0.0.1:2565/user/member-1-shard-default-config43#610970420 for shard member-1-shard-default-config43 07:02:25,654 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config43: Received RemoveServerReply{status=OK} for shard default 07:02:25,654 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config43: Leader shard successfully removed the replica shard default 07:02:25,805 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-default-config43#610970420] 07:02:25,817 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithIsolatedLeaderShard starting 07:02:25,820 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config44 07:02:25,821 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config44 07:02:25,821 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config44: Creating local shard: member-1-shard-default-config44 07:02:25,834 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:25,834 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config44 07:02:25,834 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config44: Initializing shard [default] 07:02:25,834 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-default-config44 is initialized 07:02:25,834 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config44: Received role changed for member-1-shard-default-config44 from null to IsolatedLeader 07:02:25,834 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config44: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 07:02:25,835 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config44: Scheduling 1200 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config44, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=null, activeMember=true] 07:02:27,45 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config44: Received ShardNotInitializedTimeout message for shard default 07:02:27,45 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config44: Returning NoShardLeaderException for shard default 07:02:27,46 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithIsolatedLeaderShard ending 07:02:27,148 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$y#-743248891] 07:02:27,159 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config44 07:02:27,169 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config45 07:02:27,170 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config45 07:02:27,170 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config45: Creating local shard: member-1-shard-default-config45 07:02:27,179 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:27,179 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config45 07:02:27,179 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config45: findLocalShard : non-existent 07:02:27,180 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config45: Local shard non-existent not found - shards present: [default] 07:02:27,281 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$z#-847471361] 07:02:27,291 AM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-config45 07:02:27,295 AM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-config46 07:02:27,296 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config46 07:02:27,316 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:27,316 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config46: onRemoveShardReplica: RemoveShardReplica [shardName=astronauts, memberName=MemberName{name=member-2}] 07:02:27,316 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config46: Sending RemoveServer message to peer akka://test@127.0.0.1:2565/system/testProbe-90#1207314206 for shard member-2-shard-astronauts-config46 07:02:27,327 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config46: onRemoveShardReplica: RemoveShardReplica [shardName=astronauts, memberName=MemberName{name=member-3}] 07:02:27,327 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config46: A shard replica operation for astronauts is already in progress 07:02:27,432 AM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-config47 07:02:27,433 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config47 07:02:27,433 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config47: Creating local shard: member-1-shard-default-config47 07:02:27,442 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config47: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 07:02:27,442 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config47: Returning NotInitializedException for shard default 07:02:27,544 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$A#-58837011] 07:02:27,555 AM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-config47 07:02:27,557 AM [main] [INFO] ShardManagerTest - testServerRemovedShardActorRunning starting 07:02:27,560 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config48 07:02:27,560 AM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-config48 07:02:27,561 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config48: Creating local shard: member-1-shard-default-config48 07:02:27,561 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config48: Creating local shard: member-1-shard-people-config48 07:02:27,572 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:27,572 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config48 07:02:27,572 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config48 07:02:27,573 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config48: Initializing shard [default] 07:02:27,573 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Shard member-1-shard-default-config48 is initialized 07:02:27,573 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config48: findLocalShard : people 07:02:27,573 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config48: Scheduling 600 ms timer to wait for shard ShardInformation [shardId=member-1-shard-people-config48, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 07:02:27,573 AM [test-shard-dispatcher-17] [INFO] Shard - Shard created : member-1-shard-people-config48, persistent : true 07:02:27,576 AM [test-shard-dispatcher-17] [INFO] Shard - Starting recovery for member-1-shard-people-config48 with journal batch size 1 07:02:27,579 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$k/member-1-shard-people-config48/shard-member-1:snapshot-read#-923587478] 07:02:27,579 AM [test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$k/member-1-shard-people-config48/member-1-shard-people-config48-notifier#1062309083 created and ready for shard:member-1-shard-people-config48 07:02:27,587 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config48: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 07:02:27,587 AM [test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config48: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:02:27,587 AM [test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config48: onLeadershipChange, isLeader: false, hasLeader : false 07:02:27,588 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config48: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config48 is no longer the leader 07:02:27,588 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config48: Initializing shard [people] 07:02:27,588 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Shard member-1-shard-people-config48 is initialized 07:02:27,588 AM [test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config48 , received role change from null to Follower 07:02:27,588 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Shard member-1-shard-people-config48 is initialized - notifying 1 OnShardInitialized callbacks 07:02:27,589 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config48: findLocalShard : default 07:02:27,589 AM [test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config48 , registered listener akka://test/user/$$k 07:02:27,589 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config48 : Sending Shutdown to Shard actor Actor[akka://test/user/member-1-shard-default-config48#1359086287] with 10000 ms timeout 07:02:27,592 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config48 : Local Shard replica for shard default has been removed 07:02:27,592 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config48: persisting the shard list [people] 07:02:27,592 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config48: Received role changed for member-1-shard-people-config48 from null to Follower 07:02:27,592 AM [main] [INFO] ShardManagerTest - testServerRemovedShardActorRunning ending 07:02:27,592 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config48 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 07:02:27,592 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config48: Successfully deleted prior snapshots 07:02:27,694 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-default-config48#1359086287] 07:02:27,705 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config48 : Successfully shut down Shard actor Actor[akka://test/user/member-1-shard-default-config48#1359086287] 07:02:27,705 AM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$k] 07:02:27,705 AM [test-shard-dispatcher-17] [INFO] Shard - Stopping Shard member-1-shard-people-config48 07:02:27,705 AM [test-shard-dispatcher-17] [DEBUG] MessageAssembler - member-1-shard-people-config48 (Follower): Closing 07:02:27,705 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config48: Aborting 0 pending queued transactions 07:02:27,706 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config48 07:02:27,710 AM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config49 07:02:27,711 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config49 07:02:27,711 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config49: Creating local shard: member-1-shard-default-config49 07:02:27,811 AM [main] [INFO] ShardManager - shard-manager-config49: Received role changed for member-1-shard-default-config49 from null to Follower 07:02:27,811 AM [main] [INFO] ShardManager - shard-manager-config49: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config49, leaderId=member-2-shard-default-config49, leaderPayloadVersion=13] 07:02:27,822 AM [main] [INFO] ShardManager - shard-manager-config49: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://test 07:02:27,822 AM [main] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config49 with address akka://test/user/shardmanager-config49/member-2-shard-default-config49 07:02:27,822 AM [main] [INFO] ShardManager - shard-manager-config49: All Shards are ready - data store config49 is ready 07:02:27,923 AM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$l] 07:02:27,934 AM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config49 07:02:27,936 AM [main] [INFO] ShardManagerTest - testRestoreFromSnapshot starting 07:02:27,940 AM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-config50 07:02:27,950 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config50 07:02:27,950 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config50: Restoring from ShardManagerSnapshot: ShardManagerSnapshot [ShardList = [shard1, shard2, astronauts] ] 07:02:27,950 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config50: onSnapshotOffer: ShardManagerSnapshot [ShardList = [shard1, shard2, astronauts] ] 07:02:27,950 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config50: adding shard shard1 07:02:27,951 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config50: adding shard shard2 07:02:27,951 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config50: adding shard astronauts 07:02:27,951 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config50: Creating local shard: member-1-shard-shard2-config50 07:02:27,951 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config50: Creating local shard: member-1-shard-astronauts-config50 07:02:27,951 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config50: Creating local shard: member-1-shard-shard1-config50 07:02:28,38 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:28,38 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-shard2-config50 07:02:28,38 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-astronauts-config50 07:02:28,39 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-shard1-config50 07:02:28,39 AM [test-shard-dispatcher-17] [INFO] Shard - Shard created : member-1-shard-shard2-config50, persistent : true 07:02:28,40 AM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$m/member-1-shard-shard2-config50/member-1-shard-shard2-config50-notifier#380005789 created and ready for shard:member-1-shard-shard2-config50 07:02:28,41 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$m/member-1-shard-shard2-config50/shard-member-1:snapshot-read#-324520] 07:02:28,41 AM [test-shard-dispatcher-17] [INFO] Shard - Starting recovery for member-1-shard-shard2-config50 with journal batch size 1 07:02:28,41 AM [test-shard-dispatcher-17] [INFO] Shard - Shard created : member-1-shard-shard1-config50, persistent : true 07:02:28,41 AM [test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-astronauts-config50, persistent : true 07:02:28,42 AM [test-shard-dispatcher-17] [INFO] Shard - Starting recovery for member-1-shard-shard1-config50 with journal batch size 1 07:02:28,42 AM [test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$m/member-1-shard-shard1-config50/member-1-shard-shard1-config50-notifier#-275199235 created and ready for shard:member-1-shard-shard1-config50 07:02:28,42 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$m/member-1-shard-shard1-config50/shard-member-1:snapshot-read#844628114] 07:02:28,42 AM [test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$m/member-1-shard-astronauts-config50/member-1-shard-astronauts-config50-notifier#1581961209 created and ready for shard:member-1-shard-astronauts-config50 07:02:28,44 AM [test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-astronauts-config50 with journal batch size 1 07:02:28,44 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config50: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 07:02:28,44 AM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-shard2-config50: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:02:28,44 AM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$m/member-1-shard-astronauts-config50/shard-member-1:snapshot-read#219651342] 07:02:28,44 AM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard2-config50: onLeadershipChange, isLeader: false, hasLeader : false 07:02:28,44 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config50: onStateChanged: Closing all transaction chains because shard member-1-shard-shard2-config50 is no longer the leader 07:02:28,44 AM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config50: Initializing shard [shard2] 07:02:28,44 AM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-shard2-config50 is initialized 07:02:28,44 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config50 (Follower): Received TimeoutNow - switching to Candidate 07:02:28,44 AM [test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config50 , received role change from null to Follower 07:02:28,44 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config50 (Candidate): Election: Candidate has following voting peers: [] 07:02:28,44 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config50: Set currentTerm=1, votedFor=member-1-shard-shard2-config50 07:02:28,45 AM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-shard2-config50 (Candidate): Starting new election term 1 07:02:28,45 AM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-shard2-config50 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:02:28,45 AM [test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-1-shard-shard2-config50 (Follower): Closing 07:02:28,45 AM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard2-config50: onLeadershipChange, isLeader: false, hasLeader : false 07:02:28,45 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config50: onStateChanged: Closing all transaction chains because shard member-1-shard-shard2-config50 is no longer the leader 07:02:28,45 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-shard1-config50: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 07:02:28,45 AM [test-shard-dispatcher-17] [INFO] Shard - member-1-shard-shard1-config50: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:02:28,45 AM [test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config50 , received role change from Follower to Candidate 07:02:28,45 AM [test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard1-config50: onLeadershipChange, isLeader: false, hasLeader : false 07:02:28,45 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config50: Initializing shard [shard1] 07:02:28,45 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-shard1-config50: onStateChanged: Closing all transaction chains because shard member-1-shard-shard1-config50 is no longer the leader 07:02:28,45 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-shard1-config50 is initialized 07:02:28,45 AM [test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config50 , received role change from null to Follower 07:02:28,45 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-shard1-config50 (Follower): Received TimeoutNow - switching to Candidate 07:02:28,45 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-shard1-config50 (Candidate): Election: Candidate has following voting peers: [] 07:02:28,45 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-shard1-config50: Set currentTerm=1, votedFor=member-1-shard-shard1-config50 07:02:28,45 AM [test-shard-dispatcher-17] [INFO] Shard - member-1-shard-shard1-config50 (Candidate): Starting new election term 1 07:02:28,45 AM [test-shard-dispatcher-17] [INFO] Shard - member-1-shard-shard1-config50 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:02:28,45 AM [test-shard-dispatcher-17] [DEBUG] MessageAssembler - member-1-shard-shard1-config50 (Follower): Closing 07:02:28,46 AM [test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard1-config50: onLeadershipChange, isLeader: false, hasLeader : false 07:02:28,46 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-shard1-config50: onStateChanged: Closing all transaction chains because shard member-1-shard-shard1-config50 is no longer the leader 07:02:28,46 AM [test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config50 , received role change from Follower to Candidate 07:02:28,46 AM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config50 (Candidate): Received ElectionTimeout 07:02:28,47 AM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config50 (Leader): Election: Leader has following peers: [] 07:02:28,47 AM [test-shard-dispatcher-19] [INFO] Shard - member-1-shard-shard2-config50 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 07:02:28,47 AM [test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard2-config50: onLeadershipChange, isLeader: true, hasLeader : true 07:02:28,47 AM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config50: became leader with frontend state for [] 07:02:28,47 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-shard1-config50 (Candidate): Received ElectionTimeout 07:02:28,47 AM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config50: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 07:02:28,47 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-shard1-config50 (Leader): Election: Leader has following peers: [] 07:02:28,47 AM [test-shard-dispatcher-19] [INFO] Shard - member-1-shard-astronauts-config50: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:02:28,47 AM [test-shard-dispatcher-17] [INFO] Shard - member-1-shard-shard1-config50 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 07:02:28,47 AM [test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard1-config50: onLeadershipChange, isLeader: true, hasLeader : true 07:02:28,47 AM [test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config50: onLeadershipChange, isLeader: false, hasLeader : false 07:02:28,47 AM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config50: onStateChanged: Closing all transaction chains because shard member-1-shard-astronauts-config50 is no longer the leader 07:02:28,47 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-shard1-config50: became leader with frontend state for [] 07:02:28,47 AM [test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config50 , received role change from Candidate to Leader 07:02:28,47 AM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config50 (Follower): Received TimeoutNow - switching to Candidate 07:02:28,47 AM [test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config50 , received role change from Candidate to Leader 07:02:28,47 AM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config50 (Candidate): Election: Candidate has following voting peers: [] 07:02:28,47 AM [test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config50 , registered listener akka://test/user/$$m 07:02:28,47 AM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config50: Set currentTerm=1, votedFor=member-1-shard-astronauts-config50 07:02:28,47 AM [test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config50 , registered listener akka://test/user/$$m 07:02:28,47 AM [test-shard-dispatcher-19] [INFO] Shard - member-1-shard-astronauts-config50 (Candidate): Starting new election term 1 07:02:28,47 AM [test-shard-dispatcher-19] [INFO] Shard - member-1-shard-astronauts-config50 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:02:28,47 AM [test-shard-dispatcher-19] [DEBUG] MessageAssembler - member-1-shard-astronauts-config50 (Follower): Closing 07:02:28,47 AM [test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config50: onLeadershipChange, isLeader: false, hasLeader : false 07:02:28,47 AM [test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config50 , received role change from null to Follower 07:02:28,47 AM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config50: onStateChanged: Closing all transaction chains because shard member-1-shard-astronauts-config50 is no longer the leader 07:02:28,47 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config50: Initializing shard [astronauts] 07:02:28,47 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-astronauts-config50 is initialized 07:02:28,47 AM [test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config50 , received role change from Follower to Candidate 07:02:28,47 AM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - shard-manager-config50: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-shard2-config50, leaderId=member-1-shard-shard2-config50, leaderPayloadVersion=13] 07:02:28,47 AM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - shard-manager-config50: Received role changed for member-1-shard-shard2-config50 from Candidate to Leader 07:02:28,47 AM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - shard-manager-config50: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-shard1-config50, leaderId=member-1-shard-shard1-config50, leaderPayloadVersion=13] 07:02:28,47 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50 (Candidate): Received ElectionTimeout 07:02:28,47 AM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - shard-manager-config50: Received role changed for member-1-shard-shard1-config50 from Candidate to Leader 07:02:28,47 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50 (Leader): Election: Leader has following peers: [] 07:02:28,47 AM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-astronauts-config50 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 07:02:28,47 AM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config50: onLeadershipChange, isLeader: true, hasLeader : true 07:02:28,48 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50: became leader with frontend state for [] 07:02:28,48 AM [test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config50 , received role change from Candidate to Leader 07:02:28,48 AM [test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config50 , registered listener akka://test/user/$$m 07:02:28,48 AM [test-akka.actor.default-dispatcher-22] [INFO] ShardManager - shard-manager-config50: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-astronauts-config50, leaderId=member-1-shard-astronauts-config50, leaderPayloadVersion=13] 07:02:28,48 AM [test-akka.actor.default-dispatcher-22] [INFO] ShardManager - shard-manager-config50: Received role changed for member-1-shard-astronauts-config50 from Candidate to Leader 07:02:28,48 AM [test-akka.actor.default-dispatcher-22] [INFO] ShardManager - shard-manager-config50: All Shards are ready - data store config50 is ready 07:02:28,48 AM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config50: findLocalShard : shard1 07:02:28,49 AM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config50: findLocalShard : shard2 07:02:28,49 AM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config50: findLocalShard : astronauts 07:02:28,49 AM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config50: onGetSnapshot 07:02:28,52 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config50: Expecting 3 shard snapshot replies 07:02:28,52 AM [test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-shard2-config50: onGetSnapshot 07:02:28,52 AM [test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-shard2-config50: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 07:02:28,53 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50: onGetSnapshot 07:02:28,53 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 07:02:28,53 AM [test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-shard1-config50: onGetSnapshot 07:02:28,53 AM [test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-shard1-config50: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 07:02:28,58 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 07:02:28,58 AM [test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-shard1-config50: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 07:02:28,58 AM [test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-shard2-config50: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 07:02:28,60 AM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config50: Received GetSnapshotReply [id=member-1-shard-astronauts-config50, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@11446ecc, electionTerm=1, electionVotedFor=member-1-shard-astronauts-config50, ServerConfigPayload=null]] 07:02:28,60 AM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config50: Received GetSnapshotReply [id=member-1-shard-shard1-config50, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@612f1e75, electionTerm=1, electionVotedFor=member-1-shard-shard1-config50, ServerConfigPayload=null]] 07:02:28,61 AM [test-akka.actor.default-dispatcher-20] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config50: Received GetSnapshotReply [id=member-1-shard-shard2-config50, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@6305768f, electionTerm=1, electionVotedFor=member-1-shard-shard2-config50, ServerConfigPayload=null]] 07:02:28,61 AM [test-akka.actor.default-dispatcher-20] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config50: All shard snapshots received 07:02:28,62 AM [main] [INFO] ShardManagerTest - testRestoreFromSnapshot ending 07:02:28,163 AM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$m] 07:02:28,174 AM [test-shard-dispatcher-18] [INFO] Shard - Stopping Shard member-1-shard-astronauts-config50 07:02:28,174 AM [test-shard-dispatcher-23] [INFO] Shard - Stopping Shard member-1-shard-shard1-config50 07:02:28,174 AM [test-shard-dispatcher-18] [DEBUG] MessageSlicer - member-1-shard-astronauts-config50 (Leader)_slicer-id-163: Closing 07:02:28,174 AM [test-shard-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-shard1-config50 (Leader)_slicer-id-162: Closing 07:02:28,174 AM [test-shard-dispatcher-17] [INFO] Shard - Stopping Shard member-1-shard-shard2-config50 07:02:28,174 AM [test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-shard1-config50: Aborting 0 pending queued transactions 07:02:28,174 AM [test-shard-dispatcher-17] [DEBUG] MessageSlicer - member-1-shard-shard2-config50 (Leader)_slicer-id-161: Closing 07:02:28,174 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50: Aborting 0 pending queued transactions 07:02:28,175 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-shard2-config50: Aborting 0 pending queued transactions 07:02:28,175 AM [test-akka.actor.default-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-config50 07:02:28,177 AM [main] [INFO] ShardManagerTest - testOnCreateShardWithNoInitialSchemaContext starting 07:02:28,183 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config51 07:02:28,184 AM [test-akka.actor.default-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-config51 07:02:28,194 AM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config51: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=foo-ns, moduleName=foo-module, shardName=foo, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@55516b6e] 07:02:28,195 AM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config51 doCreateShard: shardId: member-1-shard-foo-config51, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: true 07:02:28,196 AM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:28,196 AM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-foo-config51 07:02:28,196 AM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config51: findLocalShard : foo 07:02:28,197 AM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config51: Scheduling 600 ms timer to wait for shard ShardInformation [shardId=member-1-shard-foo-config51, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 07:02:28,197 AM [test-shard-dispatcher-17] [INFO] Shard - Shard created : member-1-shard-foo-config51, persistent : true 07:02:28,198 AM [test-shard-dispatcher-17] [INFO] Shard - Starting recovery for member-1-shard-foo-config51 with journal batch size 1 07:02:28,199 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$B/member-1-shard-foo-config51/shard-member-1:snapshot-read#1961238071] 07:02:28,199 AM [test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$B/member-1-shard-foo-config51/member-1-shard-foo-config51-notifier#-704635469 created and ready for shard:member-1-shard-foo-config51 07:02:28,202 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-foo-config51: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 07:02:28,202 AM [test-shard-dispatcher-17] [INFO] Shard - member-1-shard-foo-config51: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:02:28,202 AM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config51: Initializing shard [foo] 07:02:28,202 AM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-foo-config51 is initialized 07:02:28,202 AM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-foo-config51 is initialized - notifying 1 OnShardInitialized callbacks 07:02:28,202 AM [test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-foo-config51: onLeadershipChange, isLeader: false, hasLeader : false 07:02:28,202 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-foo-config51: onStateChanged: Closing all transaction chains because shard member-1-shard-foo-config51 is no longer the leader 07:02:28,202 AM [test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config51 , received role change from null to Follower 07:02:28,202 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-foo-config51 (Follower): Received TimeoutNow - switching to Candidate 07:02:28,202 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-foo-config51 (Candidate): Election: Candidate has following voting peers: [] 07:02:28,202 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-foo-config51: Set currentTerm=1, votedFor=member-1-shard-foo-config51 07:02:28,203 AM [test-shard-dispatcher-17] [INFO] Shard - member-1-shard-foo-config51 (Candidate): Starting new election term 1 07:02:28,203 AM [test-shard-dispatcher-17] [INFO] Shard - member-1-shard-foo-config51 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:02:28,203 AM [test-shard-dispatcher-17] [DEBUG] MessageAssembler - member-1-shard-foo-config51 (Follower): Closing 07:02:28,203 AM [test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-foo-config51: onLeadershipChange, isLeader: false, hasLeader : false 07:02:28,203 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-foo-config51: onStateChanged: Closing all transaction chains because shard member-1-shard-foo-config51 is no longer the leader 07:02:28,203 AM [test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config51 , received role change from Follower to Candidate 07:02:28,204 AM [main] [INFO] ShardManagerTest - testOnCreateShardWithNoInitialSchemaContext ending 07:02:28,204 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-foo-config51 (Candidate): Received ElectionTimeout 07:02:28,204 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-foo-config51 (Leader): Election: Leader has following peers: [] 07:02:28,204 AM [test-shard-dispatcher-17] [INFO] Shard - member-1-shard-foo-config51 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 07:02:28,204 AM [test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-foo-config51: onLeadershipChange, isLeader: true, hasLeader : true 07:02:28,204 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-foo-config51: became leader with frontend state for [] 07:02:28,204 AM [test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config51 , received role change from Candidate to Leader 07:02:28,204 AM [test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config51 , registered listener akka://test/user/$B 07:02:28,204 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config51: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-foo-config51, leaderId=member-1-shard-foo-config51, leaderPayloadVersion=13] 07:02:28,204 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config51: Received role changed for member-1-shard-foo-config51 from Candidate to Leader 07:02:28,204 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config51: All Shards are ready - data store config51 is ready 07:02:28,306 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$B#109040651] 07:02:28,318 AM [test-shard-dispatcher-17] [INFO] Shard - Stopping Shard member-1-shard-foo-config51 07:02:28,318 AM [test-shard-dispatcher-17] [DEBUG] MessageSlicer - member-1-shard-foo-config51 (Leader)_slicer-id-165: Closing 07:02:28,318 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-foo-config51: Aborting 0 pending queued transactions 07:02:28,318 AM [test-akka.actor.default-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-config51 07:02:28,321 AM [main] [INFO] ShardManagerTest - testGetSnapshot starting 07:02:28,324 AM [test-akka.actor.default-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-config52 07:02:28,325 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config52 07:02:28,326 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config52: Creating local shard: member-1-shard-shard2-config52 07:02:28,326 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config52: Creating local shard: member-1-shard-shard1-config52 07:02:28,434 AM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config52: onGetSnapshot 07:02:28,435 AM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:28,435 AM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-shard2-config52 07:02:28,436 AM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-shard1-config52 07:02:28,436 AM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config52: findLocalShard : shard1 07:02:28,436 AM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config52: Scheduling 600 ms timer to wait for shard ShardInformation [shardId=member-1-shard-shard1-config52, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 07:02:28,437 AM [test-shard-dispatcher-19] [INFO] Shard - Shard created : member-1-shard-shard1-config52, persistent : true 07:02:28,437 AM [test-shard-dispatcher-17] [INFO] Shard - Shard created : member-1-shard-shard2-config52, persistent : true 07:02:28,438 AM [test-shard-dispatcher-19] [INFO] Shard - Starting recovery for member-1-shard-shard1-config52 with journal batch size 1 07:02:28,438 AM [test-akka.actor.default-dispatcher-20] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$n/member-1-shard-shard1-config52/shard-member-1:snapshot-read#-302681090] 07:02:28,438 AM [test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$n/member-1-shard-shard1-config52/member-1-shard-shard1-config52-notifier#845871932 created and ready for shard:member-1-shard-shard1-config52 07:02:28,438 AM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$n/member-1-shard-shard2-config52/shard-member-1:snapshot-read#138549075] 07:02:28,438 AM [test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$n/member-1-shard-shard2-config52/member-1-shard-shard2-config52-notifier#-71112389 created and ready for shard:member-1-shard-shard2-config52 07:02:28,438 AM [test-shard-dispatcher-17] [INFO] Shard - Starting recovery for member-1-shard-shard2-config52 with journal batch size 1 07:02:28,440 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-shard1-config52: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 07:02:28,440 AM [test-shard-dispatcher-17] [INFO] Shard - member-1-shard-shard1-config52: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:02:28,440 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config52: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 07:02:28,440 AM [test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard1-config52: onLeadershipChange, isLeader: false, hasLeader : false 07:02:28,440 AM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-shard2-config52: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:02:28,440 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-shard1-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-shard1-config52 is no longer the leader 07:02:28,440 AM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard2-config52: onLeadershipChange, isLeader: false, hasLeader : false 07:02:28,440 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-shard1-config52 (Follower): Received TimeoutNow - switching to Candidate 07:02:28,440 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-shard2-config52 is no longer the leader 07:02:28,440 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-shard1-config52 (Candidate): Election: Candidate has following voting peers: [] 07:02:28,440 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-shard1-config52: Set currentTerm=1, votedFor=member-1-shard-shard1-config52 07:02:28,440 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config52 (Follower): Received TimeoutNow - switching to Candidate 07:02:28,440 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config52 (Candidate): Election: Candidate has following voting peers: [] 07:02:28,440 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config52: Set currentTerm=1, votedFor=member-1-shard-shard2-config52 07:02:28,440 AM [test-shard-dispatcher-17] [INFO] Shard - member-1-shard-shard1-config52 (Candidate): Starting new election term 1 07:02:28,440 AM [test-shard-dispatcher-17] [INFO] Shard - member-1-shard-shard1-config52 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:02:28,440 AM [test-shard-dispatcher-17] [DEBUG] MessageAssembler - member-1-shard-shard1-config52 (Follower): Closing 07:02:28,440 AM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-shard2-config52 (Candidate): Starting new election term 1 07:02:28,440 AM [test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard1-config52: onLeadershipChange, isLeader: false, hasLeader : false 07:02:28,440 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-shard1-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-shard1-config52 is no longer the leader 07:02:28,440 AM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-shard2-config52 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:02:28,440 AM [test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-1-shard-shard2-config52 (Follower): Closing 07:02:28,440 AM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard2-config52: onLeadershipChange, isLeader: false, hasLeader : false 07:02:28,440 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-shard2-config52 is no longer the leader 07:02:28,440 AM [test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config52 , received role change from null to Follower 07:02:28,440 AM [test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config52 , received role change from Follower to Candidate 07:02:28,440 AM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config52: Initializing shard [shard1] 07:02:28,440 AM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-shard1-config52 is initialized 07:02:28,440 AM [test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config52 , received role change from null to Follower 07:02:28,440 AM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-shard1-config52 is initialized - notifying 1 OnShardInitialized callbacks 07:02:28,440 AM [test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config52 , received role change from Follower to Candidate 07:02:28,441 AM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config52: Initializing shard [shard2] 07:02:28,441 AM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-shard2-config52 is initialized 07:02:28,441 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-shard1-config52 (Candidate): Received ElectionTimeout 07:02:28,441 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-shard1-config52 (Leader): Election: Leader has following peers: [] 07:02:28,441 AM [test-shard-dispatcher-17] [INFO] Shard - member-1-shard-shard1-config52 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 07:02:28,441 AM [test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard1-config52: onLeadershipChange, isLeader: true, hasLeader : true 07:02:28,441 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-shard1-config52: became leader with frontend state for [] 07:02:28,441 AM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config52: findLocalShard : shard2 07:02:28,441 AM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config52 , received role change from Candidate to Leader 07:02:28,441 AM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config52 (Candidate): Received ElectionTimeout 07:02:28,441 AM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config52 , registered listener akka://test/user/$$n 07:02:28,441 AM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config52 (Leader): Election: Leader has following peers: [] 07:02:28,441 AM [test-shard-dispatcher-19] [INFO] Shard - member-1-shard-shard2-config52 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 07:02:28,441 AM [test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard2-config52: onLeadershipChange, isLeader: true, hasLeader : true 07:02:28,442 AM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config52: became leader with frontend state for [] 07:02:28,442 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config52: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-shard1-config52, leaderId=member-1-shard-shard1-config52, leaderPayloadVersion=13] 07:02:28,442 AM [test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config52 , received role change from Candidate to Leader 07:02:28,442 AM [test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config52 , registered listener akka://test/user/$$n 07:02:28,442 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config52: Received role changed for member-1-shard-shard1-config52 from Candidate to Leader 07:02:28,442 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config52: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-shard2-config52, leaderId=member-1-shard-shard2-config52, leaderPayloadVersion=13] 07:02:28,442 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config52: Received role changed for member-1-shard-shard2-config52 from Candidate to Leader 07:02:28,442 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config52: All Shards are ready - data store config52 is ready 07:02:28,447 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config52: onGetSnapshot 07:02:28,448 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Expecting 2 shard snapshot replies 07:02:28,448 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config52: onGetSnapshot 07:02:28,448 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config52: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 07:02:28,448 AM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard1-config52: onGetSnapshot 07:02:28,448 AM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard1-config52: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 07:02:28,448 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config52: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 07:02:28,448 AM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard1-config52: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 07:02:28,448 AM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Received GetSnapshotReply [id=member-1-shard-shard2-config52, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@6d83c11c, electionTerm=1, electionVotedFor=member-1-shard-shard2-config52, ServerConfigPayload=null]] 07:02:28,448 AM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Received GetSnapshotReply [id=member-1-shard-shard1-config52, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@1778d231, electionTerm=1, electionVotedFor=member-1-shard-shard1-config52, ServerConfigPayload=null]] 07:02:28,448 AM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: All shard snapshots received 07:02:28,459 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config52: onAddShardReplica: AddShardReplica[ShardName=astronauts] 07:02:28,461 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config52: Sending AddServer message to peer akka://test@127.0.0.1:2565/system/testProbe-104#902861447 for shard member-1-shard-astronauts-config52 07:02:28,461 AM [test-shard-dispatcher-19] [INFO] Shard - Shard created : member-1-shard-astronauts-config52, persistent : true 07:02:28,461 AM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config52: Received AddServerReply{status=OK, leaderHint=} for shard astronauts from leader akka://test@127.0.0.1:2565/system/testProbe-104#902861447 07:02:28,461 AM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config52: Leader shard successfully added the replica shard astronauts 07:02:28,462 AM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-astronauts-config52 07:02:28,462 AM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config52: persisting the shard list [shard2, astronauts, shard1] 07:02:28,462 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$n/member-1-shard-astronauts-config52/shard-member-1:snapshot-read#-1482500369] 07:02:28,462 AM [test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$n/member-1-shard-astronauts-config52/member-1-shard-astronauts-config52-notifier#-808140709 created and ready for shard:member-1-shard-astronauts-config52 07:02:28,462 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config52: findLocalShard : astronauts 07:02:28,462 AM [test-shard-dispatcher-19] [INFO] Shard - Starting recovery for member-1-shard-astronauts-config52 with journal batch size 1 07:02:28,462 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config52: Scheduling 600 ms timer to wait for shard ShardInformation [shardId=member-1-shard-astronauts-config52, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 07:02:28,464 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config52 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 07:02:28,466 AM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config52: Successfully deleted prior snapshots 07:02:28,466 AM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config52: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 07:02:28,466 AM [test-shard-dispatcher-19] [INFO] Shard - member-1-shard-astronauts-config52: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:02:28,466 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config52: Initializing shard [astronauts] 07:02:28,466 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Shard member-1-shard-astronauts-config52 is initialized 07:02:28,466 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Shard member-1-shard-astronauts-config52 is initialized - notifying 1 OnShardInitialized callbacks 07:02:28,466 AM [test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config52: onLeadershipChange, isLeader: false, hasLeader : false 07:02:28,466 AM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-astronauts-config52 is no longer the leader 07:02:28,466 AM [test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config52 , received role change from null to Follower 07:02:28,466 AM [test-shard-dispatcher-19] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 07:02:28,467 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config52: onGetSnapshot 07:02:28,467 AM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config52: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 07:02:28,467 AM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config52: Re-initializing to Follower with previous leaderId null 07:02:28,467 AM [test-shard-dispatcher-19] [DEBUG] MessageAssembler - member-1-shard-astronauts-config52 (Follower): Closing 07:02:28,467 AM [test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config52: onLeadershipChange, isLeader: false, hasLeader : false 07:02:28,467 AM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-astronauts-config52 is no longer the leader 07:02:28,467 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config52: onGetSnapshot 07:02:28,467 AM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config52 (Follower): Received TimeoutNow - switching to Candidate 07:02:28,467 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Expecting 3 shard snapshot replies 07:02:28,467 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config52: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 07:02:28,467 AM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config52 (Candidate): Election: Candidate has following voting peers: [] 07:02:28,467 AM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config52: Set currentTerm=1, votedFor=member-1-shard-astronauts-config52 07:02:28,467 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-shard1-config52: onGetSnapshot 07:02:28,467 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-shard1-config52: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 07:02:28,467 AM [test-shard-dispatcher-19] [INFO] Shard - member-1-shard-astronauts-config52 (Candidate): Starting new election term 1 07:02:28,467 AM [test-shard-dispatcher-19] [INFO] Shard - member-1-shard-astronauts-config52 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 07:02:28,467 AM [test-shard-dispatcher-19] [DEBUG] MessageAssembler - member-1-shard-astronauts-config52 (Follower): Closing 07:02:28,467 AM [test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config52: onLeadershipChange, isLeader: false, hasLeader : false 07:02:28,467 AM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-astronauts-config52 is no longer the leader 07:02:28,467 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config52: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 07:02:28,467 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-shard1-config52: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 07:02:28,467 AM [test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config52 , received role change from Follower to Candidate 07:02:28,467 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Received GetSnapshotReply [id=member-1-shard-shard2-config52, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@19790a77, electionTerm=1, electionVotedFor=member-1-shard-shard2-config52, ServerConfigPayload=null]] 07:02:28,467 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Received GetSnapshotReply [id=member-1-shard-shard1-config52, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@1d7cf03e, electionTerm=1, electionVotedFor=member-1-shard-shard1-config52, ServerConfigPayload=null]] 07:02:28,468 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-astronauts-config52 (Candidate): Received ElectionTimeout 07:02:28,468 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-astronauts-config52 (Leader): Election: Leader has following peers: [] 07:02:28,468 AM [test-shard-dispatcher-17] [INFO] Shard - member-1-shard-astronauts-config52 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 07:02:28,468 AM [test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config52: onLeadershipChange, isLeader: true, hasLeader : true 07:02:28,468 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-astronauts-config52: became leader with frontend state for [] 07:02:28,468 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-astronauts-config52: onGetSnapshot 07:02:28,468 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-astronauts-config52: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 07:02:28,468 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-astronauts-config52: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 07:02:28,468 AM [test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config52 , received role change from Candidate to Leader 07:02:28,469 AM [test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config52 , registered listener akka://test/user/$$n 07:02:28,469 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config52: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-astronauts-config52, leaderId=member-1-shard-astronauts-config52, leaderPayloadVersion=13] 07:02:28,469 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Received GetSnapshotReply [id=member-1-shard-astronauts-config52, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@58920534, electionTerm=1, electionVotedFor=member-1-shard-astronauts-config52, ServerConfigPayload=null]] 07:02:28,469 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: All shard snapshots received 07:02:28,469 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config52: Received role changed for member-1-shard-astronauts-config52 from Candidate to Leader 07:02:28,469 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config52: All Shards are ready - data store config52 is ready 07:02:28,469 AM [main] [INFO] ShardManagerTest - testGetSnapshot ending 07:02:28,571 AM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$n] 07:02:28,582 AM [test-shard-dispatcher-19] [INFO] Shard - Stopping Shard member-1-shard-shard1-config52 07:02:28,582 AM [test-shard-dispatcher-17] [INFO] Shard - Stopping Shard member-1-shard-astronauts-config52 07:02:28,582 AM [test-shard-dispatcher-24] [INFO] Shard - Stopping Shard member-1-shard-shard2-config52 07:02:28,582 AM [test-shard-dispatcher-19] [DEBUG] MessageSlicer - member-1-shard-shard1-config52 (Leader)_slicer-id-168: Closing 07:02:28,582 AM [test-shard-dispatcher-17] [DEBUG] MessageSlicer - member-1-shard-astronauts-config52 (Leader)_slicer-id-171: Closing 07:02:28,582 AM [test-shard-dispatcher-24] [DEBUG] MessageSlicer - member-1-shard-shard2-config52 (Leader)_slicer-id-169: Closing 07:02:28,582 AM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard1-config52: Aborting 0 pending queued transactions 07:02:28,582 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-astronauts-config52: Aborting 0 pending queued transactions 07:02:28,582 AM [test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-shard2-config52: Aborting 0 pending queued transactions 07:02:28,582 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config52 07:02:28,585 AM [main] [INFO] ShardManagerTest - testWhenMultipleShardsPresentSyncStatusMustBeTrueForAllShards starting 07:02:28,589 AM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config53 07:02:28,591 AM [test-akka.actor.default-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-config53 07:02:28,591 AM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config53: Creating local shard: member-1-shard-default-config53 07:02:28,591 AM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config53: Creating local shard: member-1-shard-astronauts-config53 07:02:28,690 AM [main] [INFO] ShardManager - shard-manager-config53: Received role changed for member-1-shard-default-config53 from Follower to Leader 07:02:28,690 AM [main] [INFO] ShardManager - shard-manager-config53: Received role changed for member-1-shard-astronauts-config53 from Follower to Leader 07:02:28,690 AM [main] [INFO] ShardManager - shard-manager-config53: Received role changed for member-1-shard-astronauts-config53 from Leader to Follower 07:02:28,690 AM [main] [INFO] ShardManager - shard-manager-config53 Received follower initial sync status for member-1-shard-astronauts-config53 status sync done true 07:02:28,690 AM [main] [INFO] ShardManagerTest - testWhenMultipleShardsPresentSyncStatusMustBeTrueForAllShards ending 07:02:28,792 AM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$o] 07:02:28,803 AM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config53 07:02:28,805 AM [main] [INFO] ShardManagerTest - testAddShardReplicaWithFindPrimaryTimeout starting 07:02:28,808 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config54 07:02:28,809 AM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-config54 07:02:28,819 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:28,819 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config54: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://non-existent@127.0.0.1:5 07:02:28,819 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config54: All Shards are ready - data store config54 is ready 07:02:28,819 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config54: onAddShardReplica: AddShardReplica[ShardName=astronauts] 07:02:28,819 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config54: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=astronauts, waitUntilReady=true] 07:02:28,819 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config54: findPrimary for astronauts forwarding to remote ShardManager akka://non-existent@127.0.0.1:5/user/shardmanager-config54, visitedAddresses: [akka://test@127.0.0.1:2550/user/shardmanager-config54] 07:02:28,838 AM [test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://non-existent@127.0.0.1:5], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:5,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 07:02:28,838 AM [test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://non-existent@127.0.0.1:5], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:5,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 07:02:29,36 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config54: Received failure from FindPrimary for shard astronauts akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://test/user/shardmanager-config54#-418746465]] after [200 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.FindPrimary]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 07:02:29,36 AM [main] [INFO] ShardManagerTest - testAddShardReplicaWithFindPrimaryTimeout ending 07:02:29,138 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/shardmanager-config54#-418746465] 07:02:29,149 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config54 07:02:29,151 AM [main] [INFO] ShardManagerTest - testOnCreateShard starting 07:02:29,154 AM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config55 07:02:29,155 AM [test-akka.actor.default-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-config55 07:02:29,155 AM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 07:02:29,165 AM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config55: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=foo-ns, moduleName=foo-module, shardName=foo, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-5}, MemberName{name=member-6}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@5179940d] 07:02:29,165 AM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config55 doCreateShard: shardId: member-1-shard-foo-config55, memberNames: [MemberName{name=member-1}, MemberName{name=member-5}, MemberName{name=member-6}], peerAddresses: {member-6-shard-foo-config55=null, member-5-shard-foo-config55=null}, isActiveMember: true 07:02:29,166 AM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config55: findLocalShard : foo 07:02:29,166 AM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config55: Scheduling 300000 ms timer to wait for shard ShardInformation [shardId=member-1-shard-foo-config55, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 07:02:29,167 AM [test-shard-dispatcher-24] [INFO] Shard - Shard created : member-1-shard-foo-config55, persistent : false 07:02:29,168 AM [test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$C/member-1-shard-foo-config55/member-1-shard-foo-config55-notifier#-74874124 created and ready for shard:member-1-shard-foo-config55 07:02:29,168 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$C/member-1-shard-foo-config55/shard-member-1:snapshot-read#-1415601089] 07:02:29,170 AM [test-shard-dispatcher-24] [INFO] Shard - Starting recovery for member-1-shard-foo-config55 with journal batch size 1 07:02:29,172 AM [test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-foo-config55: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 07:02:29,172 AM [test-shard-dispatcher-24] [INFO] Shard - member-1-shard-foo-config55: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 07:02:29,172 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config55: Initializing shard [foo] 07:02:29,172 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Shard member-1-shard-foo-config55 is initialized 07:02:29,172 AM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Shard member-1-shard-foo-config55 is initialized - notifying 1 OnShardInitialized callbacks 07:02:29,172 AM [test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-foo-config55: onLeadershipChange, isLeader: false, hasLeader : false 07:02:29,172 AM [test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-foo-config55: onStateChanged: Closing all transaction chains because shard member-1-shard-foo-config55 is no longer the leader 07:02:29,172 AM [test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config55 , received role change from null to Follower 07:02:29,172 AM [test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config55 , registered listener akka://test/user/$C 07:02:29,172 AM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - shard-manager-config55: Received role changed for member-1-shard-foo-config55 from null to Follower 07:02:29,173 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config55: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=foo-ns, moduleName=foo-module, shardName=foo, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-5}, MemberName{name=member-6}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@5179940d] 07:02:29,173 AM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config55: Shard foo already exists 07:02:29,173 AM [main] [INFO] ShardManagerTest - testOnCreateShard ending 07:02:29,274 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$C#-789470904] 07:02:29,285 AM [test-shard-dispatcher-24] [INFO] Shard - Stopping Shard member-1-shard-foo-config55 07:02:29,285 AM [test-shard-dispatcher-24] [DEBUG] MessageAssembler - member-1-shard-foo-config55 (Follower): Closing 07:02:29,285 AM [test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-foo-config55: Aborting 0 pending queued transactions 07:02:29,285 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config55 07:02:29,286 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config1 07:02:29,286 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config35 07:02:29,287 AM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-config43 07:02:29,287 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config11 07:02:29,287 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config21 07:02:29,287 AM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-config46 07:02:29,287 AM [test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 07:02:29,287 AM [test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 07:02:29,296 AM [test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [01/22/2024 07:02:29.297] [test-akka.actor.internal-dispatcher-9] [Cluster(akka://test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config