02:55:36,765 PM [test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 02:55:36,788 PM [test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://test@127.0.0.1:2565] with UID [-263108776682355712] 02:55:36,788 PM [test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://test@127.0.0.1:2565] - Starting up, Akka version [2.6.21] ... 02:55:36,788 PM [test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://test@127.0.0.1:2565] - Registered cluster JMX MBean [akka:type=Cluster] 02:55:36,788 PM [test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://test@127.0.0.1:2565] - Started up successfully 02:55:36,804 PM [test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://test@127.0.0.1:2565] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:55:36,804 PM [test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://test@127.0.0.1:2565] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:55:36,932 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config1 02:55:36,936 PM [test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$a#1527561582] to Actor[akka://test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:55:36,987 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config1 02:55:36,987 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config1: Creating local shard: member-1-shard-default-config1 02:55:36,998 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:55:36,998 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config1 02:55:36,998 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config1: Initializing shard [default] 02:55:36,998 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config1 is initialized 02:55:36,999 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config1: Received role changed for member-1-shard-default-config1 from null to Follower 02:55:36,999 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config1: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=default, meberVotingStatusMap={member-2=true}] 02:55:37,1 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config1: findLocalShard : default 02:55:37,5 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-default-config1=true}, serversVisited=[]] to local shard akka://test/user/member-1-shard-default-config1 02:55:37,6 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config1: Received ServerChangeReply{status=NO_LEADER} from local shard akka://test/user/member-1-shard-default-config1 02:55:37,6 PM [test-akka.actor.default-dispatcher-10] [WARN] ShardManager - shard-manager-config1: ChangeServersVotingStatus failed for shard default with status NO_LEADER 02:55:37,135 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-default-config1#415317690] 02:55:37,149 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForShardLeader starting 02:55:37,151 PM [test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$b#1375444572] to Actor[akka://test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:55:37,152 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config2 02:55:37,154 PM [test-akka.actor.default-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-config2 02:55:37,154 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config2: Creating local shard: member-1-shard-default-config2 02:55:37,162 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:55:37,162 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config2 02:55:37,166 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 02:55:37,175 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config2: Scheduling 10000 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config2, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 02:55:37,314 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config2: Initializing shard [default] 02:55:37,314 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config2 is initialized 02:55:37,314 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config2 is initialized - notifying 1 OnShardInitialized callbacks 02:55:37,463 PM [test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config2: Received role changed for member-1-shard-default-config2 from Candidate to Leader 02:55:37,463 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-default-config2 is initialized - notifying 1 OnShardInitialized callbacks 02:55:37,615 PM [test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config2, leaderId=member-1-shard-default-config2, leaderPayloadVersion=13] 02:55:37,615 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-default-config2 is ready - notifying 1 OnShardInitialized callbacks 02:55:37,616 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config2: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://test@127.0.0.1:2565/user/member-1-shard-default-config#192000489, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$nA3uBRwe@2627243f] 02:55:37,617 PM [test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config2: All Shards are ready - data store config2 is ready 02:55:37,817 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForShardLeader ending 02:55:37,918 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$b#1375444572] 02:55:37,929 PM [test-akka.actor.default-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-config2 02:55:37,941 PM [test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://test/user/$$a] to Actor[akka://test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:55:37,942 PM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config3 02:55:37,944 PM [test-akka.actor.default-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-config3 02:55:37,944 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config3: Creating local shard: member-1-shard-default-config3 02:55:37,944 PM [main] [INFO] ShardManager - shard-manager-config3: Received role changed for member-1-shard-default-config3 from Follower to Candidate 02:55:37,944 PM [main] [INFO] ShardManager - shard-manager-config3 Received follower initial sync status for member-1-shard-default-config3 status sync done true 02:55:38,45 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$a] 02:55:38,56 PM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config3 02:55:38,67 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 02:55:38,79 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-504632004719091128] 02:55:38,79 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 02:55:38,95 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:55:38,95 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 02:55:38,95 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:55:38,95 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:55:38,107 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config4 02:55:38,110 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config4] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:55:38,110 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 02:55:38,110 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 02:55:38,117 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 02:55:38,120 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 02:55:38,164 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-72173478054904797] 02:55:38,165 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 02:55:38,168 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:55:38,168 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 02:55:38,175 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:55:38,175 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:55:38,183 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config4 02:55:38,183 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config4: Creating local shard: member-1-shard-default-config4 02:55:38,184 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config4: Creating local shard: member-1-shard-astronauts-config4 02:55:38,184 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config4: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:55:38,184 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-default-config4 02:55:38,184 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-astronauts-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-astronauts-config4 02:55:38,190 PM [main] [ERROR] ShardManagerTest - Mock Shard Leader Actor : TestActor[akka://cluster-test/user/member-2-shard-default-config4] 02:55:38,196 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config4 02:55:38,197 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config4] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:55:38,240 PM [main] [ERROR] ShardManagerTest - Forwarding actor : Actor[akka://cluster-test/user/shardmanager-config4/member-2-shard-default-config4#572521566] 02:55:38,243 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] ShardManager - Recovery complete : shard-manager-config4 02:55:38,243 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config4: Creating local shard: member-2-shard-default-config4 02:55:38,251 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:55:38,251 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config4 02:55:38,251 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-default-config4 02:55:38,251 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-default-config4 to akka://cluster-test/user/member-1-shard-default-config 02:55:38,252 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config4: updated peer member-1-shard-default-config4 on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-default-config4 on shard member-1-shard-default-config4 whose actor address is Actor[akka://cluster-test/user/member-1-shard-default-config#1245551655] 02:55:38,252 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-astronauts-config4 02:55:38,252 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-astronauts-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-astronauts-config4 02:55:38,252 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-astronauts-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-astronauts-config4 to akka://cluster-test/user/member-1-shard-default-config 02:55:38,252 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config4: updated peer member-1-shard-astronauts-config4 on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-astronauts-config4 on shard member-1-shard-astronauts-config4 whose actor address is Actor[akka://cluster-test/user/member-1-shard-default-config#1245551655] 02:55:38,252 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config4: Initializing shard [default] 02:55:38,252 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-default-config4 is initialized 02:55:38,252 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config4: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config4, leaderId=member-2-shard-default-config4, leaderPayloadVersion=12] 02:55:38,252 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config4: Received role changed for member-1-shard-default-config4 from Candidate to Follower 02:55:38,253 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:55:38,253 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-2-shard-default-config4 02:55:38,253 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config4: Initializing shard [default] 02:55:38,253 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardInformation - Shard member-2-shard-default-config4 is initialized 02:55:38,253 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config4: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-config4, leaderId=member-2-shard-default-config4, leaderPayloadVersion=12] 02:55:38,253 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config4: Received role changed for member-2-shard-default-config4 from Candidate to Leader 02:55:38,253 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config4: All Shards are ready - data store config4 is ready 02:55:38,535 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 02:55:38,585 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 02:55:38,585 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config4: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:55:38,585 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-default-config4 02:55:38,585 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-default-config4 to akka://cluster-test/user/member-2-shard-default-config4 02:55:38,586 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config4: All Shards are ready - data store config4 is ready 02:55:39,112 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 02:55:39,112 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config4: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:55:39,112 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config4 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-default-config4 02:55:39,112 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config4 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-default-config4 to akka://cluster-test/user/member-1-shard-default-config 02:55:39,112 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-astronauts-config4 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-astronauts-config4 02:55:39,112 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-astronauts-config4 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-astronauts-config4 to akka://cluster-test/user/member-1-shard-default-config 02:55:39,113 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config4: onRemoveShardReplica: RemoveShardReplica [shardName=default, memberName=MemberName{name=member-1}] 02:55:39,115 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config4: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 02:55:39,116 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config4: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-default-config4, primaryVersion=12] 02:55:39,116 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config4: Sending RemoveServer message to peer akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-default-config4 for shard member-1-shard-default-config4 02:55:39,139 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config4: Received RemoveServerReply{status=OK, leaderHint=member-2-shard-default-config4} for shard default 02:55:39,139 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config4: Leader shard successfully removed the replica shard default 02:55:39,209 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config4: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:55:39,209 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config4 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-default-config4 02:55:39,209 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config4 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-default-config4 to akka://cluster-test/user/member-2-shard-default-config4 02:55:39,209 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config4: All Shards are ready - data store config4 is ready 02:55:39,265 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config4 02:55:39,266 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:55:39,266 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:55:39,297 PM [cluster-test-akka.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:55:39,297 PM [cluster-test-akka.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:55:39,301 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 02:55:39,328 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - Stopping ShardManager shard-manager-config4 02:55:39,329 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:55:39,329 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:55:40,359 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 14:55:40.361] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:55:40,380 PM [test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$c#1297150879] to Actor[akka://test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:55:40,381 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config5 02:55:40,382 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config5 02:55:40,382 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config5: Creating local shard: member-1-shard-default-config5 02:55:40,395 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:55:40,395 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config5 02:55:40,395 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config5: Initializing shard [default] 02:55:40,395 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config5 is initialized 02:55:40,548 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$c#1297150879] 02:55:40,560 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config5 02:55:40,564 PM [test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://test/user/$$b] to Actor[akka://test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:55:40,564 PM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config6 02:55:40,565 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config6 02:55:40,565 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config6: Creating local shard: member-1-shard-default-config6 02:55:40,571 PM [main] [INFO] ShardManager - shard-manager-config6: Received role changed for member-1-shard-default-config6 from null to Follower 02:55:40,581 PM [main] [INFO] ShardManager - shard-manager-config6: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://test 02:55:40,582 PM [main] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config6 with address akka://test/user/shardmanager-config6/member-2-shard-default-config6 02:55:40,582 PM [main] [INFO] ShardManager - shard-manager-config6: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config6, leaderId=member-2-shard-default-config6, leaderPayloadVersion=13] 02:55:40,582 PM [main] [INFO] ShardManager - shard-manager-config6: All Shards are ready - data store config6 is ready 02:55:40,683 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$b] 02:55:40,694 PM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config6 02:55:40,700 PM [test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$d#764196442] to Actor[akka://test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:55:40,700 PM [test-akka.actor.default-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-config7 02:55:40,701 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config7 02:55:40,711 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config7: onRemoveShardReplica: RemoveShardReplica [shardName=model-inventory, memberName=MemberName{name=member-1}] 02:55:40,711 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config7: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=model-inventory, waitUntilReady=true] 02:55:40,711 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config7: No shard found for model-inventory 02:55:40,712 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config7: Failed to find leader for shard model-inventory: received response: {} org.opendaylight.controller.cluster.datastore.exceptions.PrimaryNotFoundException: No primary shard found for model-inventory. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.findPrimary(ShardManager.java:1051) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:207) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManagerTest$TestShardManager.handleCommand(ShardManagerTest.java:2198) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 02:55:40,814 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$d#764196442] 02:55:40,825 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-config7 02:55:40,827 PM [main] [INFO] ShardManagerTest - testAddShardReplicaWithPreExistingLocalReplicaLeader starting 02:55:40,834 PM [test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$e#-1371463003] to Actor[akka://test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:55:40,835 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config8 02:55:40,835 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config8 02:55:40,835 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config8: Creating local shard: member-1-shard-default-config8 02:55:40,845 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:55:40,845 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config8 02:55:40,845 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config8: Initializing shard [default] 02:55:40,845 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-default-config8 is initialized 02:55:40,845 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config8: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config8, leaderId=member-1-shard-default-config8, leaderPayloadVersion=13] 02:55:40,846 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config8: Received role changed for member-1-shard-default-config8 from Candidate to Leader 02:55:40,846 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config8: All Shards are ready - data store config8 is ready 02:55:40,846 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config8: onAddShardReplica: AddShardReplica[ShardName=default] 02:55:40,848 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config8: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 02:55:40,848 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config8: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://test@127.0.0.1:2565/user/member-1-shard-default-config#-290079637, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$nA3uBRwe@4246b98b] 02:55:40,848 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config8: Local shard default already exists 02:55:40,848 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config8: findLocalShard : default 02:55:40,849 PM [main] [INFO] ShardManagerTest - testAddShardReplicaWithPreExistingLocalReplicaLeader ending 02:55:40,950 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$e#-1371463003] 02:55:40,961 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-config8 02:55:40,963 PM [main] [INFO] ShardManagerTest - testOnReceiveFindLocalShardWaitForShardInitialized starting 02:55:40,966 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config9 02:55:40,966 PM [test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$f#-702634125] to Actor[akka://test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:55:40,967 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config9 02:55:40,967 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config9: Creating local shard: member-1-shard-default-config9 02:55:40,976 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:55:40,976 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config9 02:55:40,976 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config9: findLocalShard : default 02:55:40,976 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config9: Scheduling 600 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config9, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 02:55:40,977 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config9: Initializing shard [default] 02:55:40,977 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-default-config9 is initialized 02:55:40,977 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-default-config9 is initialized - notifying 1 OnShardInitialized callbacks 02:55:40,979 PM [main] [INFO] ShardManagerTest - testOnReceiveFindLocalShardWaitForShardInitialized starting 02:55:41,81 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$f#-702634125] 02:55:41,92 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Stopping ShardManager shard-manager-config9 02:55:41,93 PM [main] [INFO] ShardManagerTest - testShardAvailabilityChangeOnMemberUnreachableAndLeadershipChange starting 02:55:41,102 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 02:55:41,114 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [1472300752209891081] 02:55:41,118 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 02:55:41,118 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 02:55:41,118 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 02:55:41,118 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:55:41,118 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:55:41,131 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config10] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:55:41,133 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config10 02:55:41,133 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 02:55:41,133 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 02:55:41,133 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 02:55:41,146 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shard-manager-config10 02:55:41,147 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config10: Creating local shard: member-1-shard-default-config10 02:55:41,147 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config10: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:55:41,147 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config10 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config10/member-1-shard-default-config10 02:55:41,166 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 02:55:41,179 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-7506010929937184215] 02:55:41,179 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 02:55:41,186 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:55:41,186 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 02:55:41,208 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:55:41,208 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:55:41,212 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config10 02:55:41,219 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:55:41,219 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config10 02:55:41,219 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config10 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config10/member-1-shard-default-config10 02:55:41,219 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config10 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config10/member-1-shard-default-config10 to akka://cluster-test/user/member-1-shard-default-config 02:55:41,219 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config10: updated peer member-1-shard-default-config10 on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config10/member-1-shard-default-config10 on shard member-1-shard-default-config10 whose actor address is Actor[akka://cluster-test/user/member-1-shard-default-config#-515285260] 02:55:41,219 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config10: Initializing shard [default] 02:55:41,219 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-default-config10 is initialized 02:55:41,219 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config10: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config10, leaderId=member-2-shard-default-config10, leaderPayloadVersion=13] 02:55:41,219 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config10: Received role changed for member-1-shard-default-config10 from Candidate to Follower 02:55:41,280 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-config10 02:55:41,281 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config10: Creating local shard: member-2-shard-default-config10 02:55:41,281 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:55:41,281 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-2-shard-default-config10 02:55:41,281 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config10: Initializing shard [default] 02:55:41,281 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardInformation - Shard member-2-shard-default-config10 is initialized 02:55:41,281 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config10: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-config10, leaderId=member-2-shard-default-config10, leaderPayloadVersion=13] 02:55:41,281 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config10: Received role changed for member-2-shard-default-config10 from Candidate to Leader 02:55:41,281 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config10: All Shards are ready - data store config10 is ready 02:55:41,311 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 02:55:41,321 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 02:55:41,321 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config10: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:55:41,321 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config10 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config10/member-1-shard-default-config10 02:55:41,321 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config10 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config10/member-1-shard-default-config10 to akka://cluster-test/user/member-2-shard-default-config 02:55:41,321 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config10: All Shards are ready - data store config10 is ready 02:55:42,131 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 02:55:42,131 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config10: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:55:42,132 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config10 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config10/member-2-shard-default-config10 02:55:42,132 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config10 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config10/member-2-shard-default-config10 to akka://cluster-test/user/member-1-shard-default-config 02:55:42,132 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config10: All Shards are ready - data store config10 is ready 02:55:42,133 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config10: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 02:55:42,133 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config10: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config10/member-2-shard-default-config10, primaryVersion=13] 02:55:42,134 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 02:55:42,134 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Marking Leader member-2-shard-default-config10 as unavailable. 02:55:42,138 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config10: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 02:55:42,138 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config10: Scheduling 1200 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config10, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=member-2-shard-default-config10, activeMember=true] 02:55:42,213 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config10: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:55:42,213 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config10 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config10/member-2-shard-default-config10 02:55:42,213 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config10 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config10/member-2-shard-default-config10 to akka://cluster-test/user/member-2-shard-default-config 02:55:42,213 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config10: All Shards are ready - data store config10 is ready 02:55:43,349 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config10: Received ShardNotInitializedTimeout message for shard default 02:55:43,349 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config10: Returning NoShardLeaderException for shard default 02:55:43,350 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config10: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config10, leaderId=member-1-shard-default-config10, leaderPayloadVersion=13] 02:55:43,351 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config10: All Shards are ready - data store config10 is ready 02:55:43,351 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config10: Received role changed for member-1-shard-default-config10 from Follower to Leader 02:55:43,351 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config10: All Shards are ready - data store config10 is ready 02:55:43,351 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config10: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 02:55:43,351 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config10: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/member-1-shard-default-config#-515285260, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$nA3uBRwe@3f405c64] 02:55:43,351 PM [main] [INFO] ShardManagerTest - testShardAvailabilityChangeOnMemberUnreachableAndLeadershipChange ending 02:55:43,453 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config10 02:55:43,454 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:55:43,454 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:55:43,462 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:55:43,463 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:55:43,465 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 02:55:43,486 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-config10 02:55:43,487 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:55:43,487 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:55:44,506 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 14:55:44.508] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:55:44,519 PM [test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://test/user/shardmanager-config11] to Actor[akka://test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:55:44,520 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Starting ShardManager shard-manager-config11 02:55:44,521 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config11 02:55:44,541 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:55:44,541 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config11: onAddShardReplica: AddShardReplica[ShardName=astronauts] 02:55:44,543 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config11: Sending AddServer message to peer akka://test@127.0.0.1:2565/system/testProbe-530#828355955 for shard member-1-shard-astronauts-config11 02:55:44,554 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config11: onRemoveShardReplica: RemoveShardReplica [shardName=astronauts, memberName=MemberName{name=member-2}] 02:55:44,555 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config11: A shard replica operation for astronauts is already in progress 02:55:44,659 PM [main] [INFO] ShardManagerTest - testPerShardDatastoreContext starting 02:55:44,666 PM [test-akka.actor.default-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-config12 02:55:44,668 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config12 02:55:44,669 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config12: Creating local shard: member-1-shard-default-config12 02:55:44,669 PM [test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$g#-2091007727] to Actor[akka://test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:55:44,669 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config12: Creating local shard: member-1-shard-topology-config12 02:55:44,679 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:55:44,679 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config12 02:55:44,679 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-topology-config12 02:55:44,680 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-default-config12 02:55:44,680 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-topology-config12 02:55:44,730 PM [main] [INFO] ShardManagerTest - testPerShardDatastoreContext ending 02:55:44,832 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/default1#-1688823368] 02:55:44,843 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/topology2#-397720855] 02:55:44,843 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$g#-2091007727] 02:55:44,843 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-config12 02:55:44,845 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForLocalLeaderShard starting 02:55:44,848 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config13 02:55:44,849 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config13 02:55:44,849 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config13: Creating local shard: member-1-shard-default-config13 02:55:44,858 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:55:44,858 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config13 02:55:44,858 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config13: Initializing shard [default] 02:55:44,859 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-default-config13 is initialized 02:55:44,859 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config13: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config13, leaderId=member-1-shard-default-config13, leaderPayloadVersion=13] 02:55:44,909 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config13: Received role changed for member-1-shard-default-config13 from Candidate to Leader 02:55:44,909 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config13: All Shards are ready - data store config13 is ready 02:55:44,909 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config13: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 02:55:44,909 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config13: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://test@127.0.0.1:2565/user/member-1-shard-default-config#-1634125218, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$nA3uBRwe@5c817891] 02:55:44,910 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForLocalLeaderShard ending 02:55:45,11 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$h#-1215260990] 02:55:45,22 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config13 02:55:45,24 PM [main] [INFO] ShardManagerTest - testShardAvailabilityChangeOnMemberWithNameContainedInLeaderIdUnreachable starting 02:55:45,32 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 02:55:45,39 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2562] with UID [-7938507214959321756] 02:55:45,39 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Starting up, Akka version [2.6.21] ... 02:55:45,46 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Registered cluster JMX MBean [akka:type=Cluster] 02:55:45,46 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Started up successfully 02:55:45,51 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:55:45,51 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:55:45,52 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Node [akka://cluster-test@127.0.0.1:2562] is JOINING itself (with roles [member-256, dc-default], version [0.0.0]) and forming new cluster 02:55:45,52 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - is the new leader among reachable nodes (more leaders may exist) 02:55:45,52 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Leader is moving node [akka://cluster-test@127.0.0.1:2562] to [Up] 02:55:45,58 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config14 02:55:45,59 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config14] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:55:45,83 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config14 02:55:45,84 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config14: Creating local shard: member-256-shard-default-config14 02:55:45,85 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config14: Received MemberUp: memberName: MemberName{name=member-256}, address: akka://cluster-test@127.0.0.1:2562 02:55:45,85 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-256-shard-default-config14 with address akka://cluster-test@127.0.0.1:2562/user/shardmanager-config14/member-256-shard-default-config14 02:55:45,98 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 02:55:45,107 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [3604799925211236150] 02:55:45,107 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 02:55:45,112 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:55:45,113 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 02:55:45,113 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:55:45,113 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:55:45,134 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config14] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:55:45,136 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Starting ShardManager shard-manager-config14 02:55:45,137 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:55:45,137 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-256-shard-default-config14 02:55:45,137 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-256-shard-default-config14 with address akka://cluster-test@127.0.0.1:2562/user/shardmanager-config14/member-256-shard-default-config14 02:55:45,137 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-256-shard-default-config14 with address akka://cluster-test@127.0.0.1:2562/user/shardmanager-config14/member-256-shard-default-config14 to akka://cluster-test/user/member-256-shard-default-config 02:55:45,137 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config14: updated peer member-256-shard-default-config14 on member MemberName{name=member-256} with address akka://cluster-test@127.0.0.1:2562/user/shardmanager-config14/member-256-shard-default-config14 on shard member-256-shard-default-config14 whose actor address is Actor[akka://cluster-test/user/member-256-shard-default-config#991393071] 02:55:45,137 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config14: Initializing shard [default] 02:55:45,137 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-256-shard-default-config14 is initialized 02:55:45,137 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config14: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-256-shard-default-config14, leaderId=member-256-shard-default-config14, leaderPayloadVersion=13] 02:55:45,137 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config14: Received role changed for member-256-shard-default-config14 from Candidate to Leader 02:55:45,137 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config14: All Shards are ready - data store config14 is ready 02:55:45,149 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config14 02:55:45,149 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config14: Creating local shard: member-2-shard-default-config14 02:55:45,150 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:55:45,150 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-2-shard-default-config14 02:55:45,150 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config14: Initializing shard [default] 02:55:45,150 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Shard member-2-shard-default-config14 is initialized 02:55:45,150 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config14: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-config14, leaderId=member-256-shard-default-config14, leaderPayloadVersion=13] 02:55:45,150 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config14: Received role changed for member-2-shard-default-config14 from Candidate to Follower 02:55:45,202 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 02:55:45,211 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2562] 02:55:45,211 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config14: Received MemberUp: memberName: MemberName{name=member-256}, address: akka://cluster-test@127.0.0.1:2562 02:55:45,211 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-256-shard-default-config14 with address akka://cluster-test@127.0.0.1:2562/user/shardmanager-config14/member-256-shard-default-config14 02:55:45,211 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-256-shard-default-config14 with address akka://cluster-test@127.0.0.1:2562/user/shardmanager-config14/member-256-shard-default-config14 to akka://cluster-test/user/member-2-shard-default-config 02:55:45,211 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config14: All Shards are ready - data store config14 is ready 02:55:46,72 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 02:55:46,75 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config14: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:55:46,75 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config14 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config14/member-2-shard-default-config14 02:55:46,75 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config14 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config14/member-2-shard-default-config14 to akka://cluster-test/user/member-256-shard-default-config 02:55:46,75 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config14: All Shards are ready - data store config14 is ready 02:55:46,75 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config14: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 02:55:46,75 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config14: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2562/user/member-256-shard-default-config#991393071, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$nA3uBRwe@7c3f181b] 02:55:46,76 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 02:55:46,76 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config14: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 02:55:46,76 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config14: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2562/user/member-256-shard-default-config#991393071, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$nA3uBRwe@7c3f181b] 02:55:46,78 PM [main] [INFO] ShardManagerTest - testShardAvailabilityChangeOnMemberWithNameContainedInLeaderIdUnreachable ending 02:55:46,139 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config14: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:55:46,139 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config14 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config14/member-2-shard-default-config14 02:55:46,139 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config14 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config14/member-2-shard-default-config14 to akka://cluster-test/user/member-2-shard-default-config 02:55:46,139 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config14: All Shards are ready - data store config14 is ready 02:55:46,180 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-config14 02:55:46,181 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:55:46,181 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:55:46,190 PM [cluster-test-akka.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:55:46,190 PM [cluster-test-akka.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:55:46,192 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 02:55:46,202 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-config14 02:55:46,203 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:55:46,203 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:55:47,148 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - is the new leader among reachable nodes (more leaders may exist) [WARN] [03/11/2024 14:55:47.225] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:55:47,246 PM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config15 02:55:47,247 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config15 02:55:47,247 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config15: Creating local shard: member-1-shard-default-config15 02:55:47,248 PM [main] [INFO] ShardManager - shard-manager-config15: Received role changed for member-1-shard-default-config15 from Follower to Leader 02:55:47,349 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$c] 02:55:47,359 PM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config15 02:55:47,364 PM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config16 02:55:47,365 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config16 02:55:47,365 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config16: Creating local shard: member-1-shard-default-config16 02:55:47,366 PM [main] [INFO] ShardManager - shard-manager-config16: Received role changed for member-1-shard-default-config16 from Candidate to Leader 02:55:47,366 PM [main] [INFO] ShardManager - shard-manager-config16: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config16, leaderId=member-1-shard-default-config16, leaderPayloadVersion=13] 02:55:47,366 PM [main] [INFO] ShardManager - shard-manager-config16: All Shards are ready - data store config16 is ready 02:55:47,467 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$d] 02:55:47,478 PM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config16 02:55:47,480 PM [main] [INFO] ShardManagerTest - testOnCreateShardWithLocalMemberNotInShardConfig starting 02:55:47,482 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config17 02:55:47,484 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config17 02:55:47,484 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:55:47,496 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config17: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=foo-ns, moduleName=foo-module, shardName=foo, shardMemberNames=[MemberName{name=member-5}, MemberName{name=member-6}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@53dd4107] 02:55:47,498 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config17 doCreateShard: shardId: member-1-shard-foo-config17, memberNames: [MemberName{name=member-5}, MemberName{name=member-6}], peerAddresses: {}, isActiveMember: false 02:55:47,501 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config17: findLocalShard : foo 02:55:47,501 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config17: Scheduling 60000 ms timer to wait for shard ShardInformation [shardId=member-1-shard-foo-config17, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=false] 02:55:47,512 PM [test-shard-dispatcher-16] [INFO] Shard - Shard created : member-1-shard-foo-config17, persistent : true 02:55:47,525 PM [test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$i/member-1-shard-foo-config17/member-1-shard-foo-config17-notifier#1004718100 created and ready for shard:member-1-shard-foo-config17 02:55:47,531 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$i/member-1-shard-foo-config17/shard-member-1:snapshot-read#627729114] 02:55:47,535 PM [test-shard-dispatcher-16] [INFO] Shard - Starting recovery for member-1-shard-foo-config17 with journal batch size 1 02:55:47,547 PM [test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-foo-config17: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 02:55:47,550 PM [test-shard-dispatcher-16] [INFO] Shard - member-1-shard-foo-config17: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:55:47,550 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config17: Initializing shard [foo] 02:55:47,550 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-foo-config17 is initialized 02:55:47,550 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-foo-config17 is initialized - notifying 1 OnShardInitialized callbacks 02:55:47,550 PM [main] [INFO] ShardManagerTest - testOnCreateShardWithLocalMemberNotInShardConfig ending 02:55:47,556 PM [test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-foo-config17: onLeadershipChange, isLeader: false, hasLeader : false 02:55:47,556 PM [test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-foo-config17: onStateChanged: Closing all transaction chains because shard member-1-shard-foo-config17 is no longer the leader 02:55:47,557 PM [test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config17 , received role change from null to Follower 02:55:47,560 PM [test-shard-dispatcher-16] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 02:55:47,561 PM [test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config17 , registered listener akka://test/user/$i 02:55:47,562 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config17: Received role changed for member-1-shard-foo-config17 from null to Follower 02:55:47,652 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$i#-766675363] 02:55:47,671 PM [test-shard-dispatcher-16] [INFO] Shard - Stopping Shard member-1-shard-foo-config17 02:55:47,671 PM [test-shard-dispatcher-16] [DEBUG] MessageAssembler - member-1-shard-foo-config17 (Follower): Closing 02:55:47,672 PM [test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-foo-config17: Aborting 0 pending queued transactions 02:55:47,672 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Stopping ShardManager shard-manager-config17 02:55:47,674 PM [main] [INFO] ShardManagerTest - testServerRemovedShardActorNotRunning starting 02:55:47,676 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Starting ShardManager shard-manager-config18 02:55:47,678 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config18 02:55:47,678 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config18: Creating local shard: member-1-shard-default-config18 02:55:47,679 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config18: Creating local shard: member-1-shard-people-config18 02:55:47,787 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config18: findLocalShard : people 02:55:47,787 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config18: Returning NotInitializedException for shard people 02:55:47,787 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config18: findLocalShard : default 02:55:47,787 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config18: Returning NotInitializedException for shard default 02:55:47,789 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config18 : Local Shard replica for shard default has been removed 02:55:47,789 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config18: persisting the shard list [people] 02:55:47,789 PM [main] [INFO] ShardManagerTest - testServerRemovedShardActorNotRunning ending 02:55:47,798 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config18 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 02:55:47,805 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config18: Successfully deleted prior snapshots 02:55:47,891 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$e] 02:55:47,901 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Stopping ShardManager shard-manager-config18 02:55:47,906 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config19 02:55:47,907 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config19 02:55:47,907 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config19: Creating local shard: member-1-shard-default-config19 02:55:47,917 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:55:47,917 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config19 02:55:47,917 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config19: Initializing shard [default] 02:55:47,917 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config19 is initialized 02:55:47,917 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config19: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 02:55:47,917 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config19: Returning NoShardLeaderException for shard default 02:55:48,19 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$j#1795386334] 02:55:48,30 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config19 02:55:48,32 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForFollowerShardWithNoInitialLeaderId starting 02:55:48,34 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config20 02:55:48,35 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config20 02:55:48,35 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config20: Creating local shard: member-1-shard-default-config20 02:55:48,46 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:55:48,46 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config20 02:55:48,46 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config20: Initializing shard [default] 02:55:48,46 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardInformation - Shard member-1-shard-default-config20 is initialized 02:55:48,46 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - shard-manager-config20: Received role changed for member-1-shard-default-config20 from Candidate to Follower 02:55:48,46 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config20: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 02:55:48,46 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config20: Returning NoShardLeaderException for shard default 02:55:48,47 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config20: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config20, leaderId=member-1-shard-default-config20, leaderPayloadVersion=13] 02:55:48,47 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config20: All Shards are ready - data store config20 is ready 02:55:48,47 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config20: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 02:55:48,47 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config20: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://test@127.0.0.1:2565/user/member-1-shard-default-config#-880806557, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$nA3uBRwe@1a762d09] 02:55:48,47 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForFollowerShardWithNoInitialLeaderId starting 02:55:48,149 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$k#-2012402870] 02:55:48,160 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config20 02:55:48,164 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config21 02:55:48,165 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config21 02:55:48,184 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:55:48,184 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config21: onAddShardReplica: AddShardReplica[ShardName=astronauts] 02:55:48,185 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config21: Sending AddServer message to peer akka://test@127.0.0.1:2565/system/testProbe-548#147986857 for shard member-1-shard-astronauts-config21 02:55:48,196 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config21: onAddShardReplica: AddShardReplica[ShardName=astronauts] 02:55:48,196 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config21: A shard replica operation for astronauts is already in progress 02:55:48,299 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithCandidateShard starting 02:55:48,302 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Starting ShardManager shard-manager-config22 02:55:48,306 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config22 02:55:48,306 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config22: Creating local shard: member-1-shard-default-config22 02:55:48,312 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:55:48,312 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config22 02:55:48,313 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config22: Initializing shard [default] 02:55:48,313 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-default-config22 is initialized 02:55:48,313 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config22: Received role changed for member-1-shard-default-config22 from null to Candidate 02:55:48,313 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config22: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 02:55:48,313 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config22: Scheduling 1200 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config22, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Candidate, leaderId=null, activeMember=true] 02:55:49,524 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config22: Received ShardNotInitializedTimeout message for shard default 02:55:49,524 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config22: Returning NoShardLeaderException for shard default 02:55:49,524 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithCandidateShard ending 02:55:49,626 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$l#-1597906104] 02:55:49,636 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-config22 02:55:49,638 PM [main] [INFO] ShardManagerTest - testAddShardReplicaWithAddServerReplyFailure starting 02:55:49,642 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Starting ShardManager shard-manager-config23 02:55:49,643 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config23 02:55:49,761 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:55:49,772 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config23: onAddShardReplica: AddShardReplica[ShardName=astronauts] 02:55:49,773 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config23: Sending AddServer message to peer akka://test@127.0.0.1:2565/system/testProbe-553#10971006 for shard member-1-shard-astronauts-config23 02:55:49,774 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config23: Received AddServerReply{status=TIMEOUT} for shard astronauts from leader akka://test@127.0.0.1:2565/system/testProbe-553#10971006 02:55:49,774 PM [test-akka.actor.default-dispatcher-6] [WARN] ShardManager - shard-manager-config23: Leader failed to add shard replica astronauts with status TIMEOUT 02:55:49,776 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config23: findLocalShard : astronauts 02:55:49,776 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config23: Local shard astronauts not found - shards present: [] 02:55:49,776 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config23: onAddShardReplica: AddShardReplica[ShardName=astronauts] 02:55:49,777 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config23: Sending AddServer message to peer akka://test@127.0.0.1:2565/system/testProbe-553#10971006 for shard member-1-shard-astronauts-config23 02:55:49,777 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config23: Received AddServerReply{status=NO_LEADER} for shard astronauts from leader akka://test@127.0.0.1:2565/system/testProbe-553#10971006 02:55:49,777 PM [test-akka.actor.default-dispatcher-13] [WARN] ShardManager - shard-manager-config23: Leader failed to add shard replica astronauts with status NO_LEADER 02:55:49,778 PM [main] [INFO] ShardManagerTest - testAddShardReplicaWithAddServerReplyFailure ending 02:55:49,879 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-astronauts-config#-1858979978] 02:55:49,890 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/shardmanager-config23] 02:55:49,890 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Stopping ShardManager shard-manager-config23 02:55:49,894 PM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config24 02:55:49,895 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config24 02:55:49,895 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config24: Creating local shard: member-1-shard-default-config24 02:55:49,994 PM [main] [INFO] ShardManager - shard-manager-config24: Received role changed for member-1-shard-default-config24 from Candidate to Follower 02:55:49,994 PM [main] [INFO] ShardManager - shard-manager-config24 Received follower initial sync status for member-1-shard-default-config24 status sync done true 02:55:49,994 PM [main] [INFO] ShardManager - shard-manager-config24 Received follower initial sync status for member-1-shard-default-config24 status sync done false 02:55:50,96 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$f] 02:55:50,106 PM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config24 02:55:50,110 PM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config25 02:55:50,112 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config25 02:55:50,112 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config25: Creating local shard: member-1-shard-default-config25 02:55:50,113 PM [main] [INFO] ShardManager - shard-manager-config25: Received role changed for unknown from Candidate to Leader 02:55:50,214 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$g] 02:55:50,225 PM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config25 02:55:50,227 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithUninitializedShard starting 02:55:50,229 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config26 02:55:50,239 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config26 02:55:50,239 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config26: Creating local shard: member-1-shard-default-config26 02:55:50,240 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:55:50,240 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config26 02:55:50,240 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config26: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 02:55:50,240 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config26: Scheduling 600 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config26, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 02:55:50,853 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config26: Received ShardNotInitializedTimeout message for shard default 02:55:50,853 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config26: Returning NotInitializedException for shard default 02:55:50,854 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config26: Initializing shard [default] 02:55:50,854 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardInformation - Shard member-1-shard-default-config26 is initialized 02:55:51,53 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithUninitializedShard ending 02:55:51,155 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$m#1380410847] 02:55:51,166 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Stopping ShardManager shard-manager-config26 02:55:51,171 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config27 02:55:51,173 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config27 02:55:51,173 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config27: Creating local shard: member-1-shard-default-config27 02:55:51,182 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:55:51,182 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config27 02:55:51,182 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config27: Initializing shard [default] 02:55:51,182 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-default-config27 is initialized 02:55:51,182 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config27: findLocalShard : default 02:55:51,284 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$n#1210135705] 02:55:51,294 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-config27 02:55:51,296 PM [main] [INFO] ShardManagerTest - testRegisterForShardLeaderChanges starting 02:55:51,298 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config28 02:55:51,299 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config28 02:55:51,300 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config28: Creating local shard: member-1-shard-default-config28 02:55:51,309 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:55:51,309 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config28 02:55:51,309 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config28: Initializing shard [default] 02:55:51,309 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config28 is initialized 02:55:51,309 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config28: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@13072dfc 02:55:51,318 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config28: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config28, leaderId=member-1-shard-default-config28, leaderPayloadVersion=13] 02:55:51,336 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config28: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config28, leaderId=member-1-shard-default-config28, leaderPayloadVersion=13] 02:55:51,837 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config28: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config28, leaderId=null, leaderPayloadVersion=13] 02:55:51,848 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config28: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config28, leaderId=member-2-shard-default-config28, leaderPayloadVersion=13] 02:55:51,859 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config28: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config28, leaderId=member-1-shard-default-config28, leaderPayloadVersion=13] 02:55:52,359 PM [main] [INFO] ShardManagerTest - testRegisterForShardLeaderChanges ending 02:55:52,460 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$o#-116646060] 02:55:52,471 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-config28 02:55:52,476 PM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config29 02:55:52,478 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config29 02:55:52,478 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config29: Creating local shard: member-1-shard-default-config29 02:55:52,579 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$h] 02:55:52,590 PM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config29 02:55:52,594 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config30 02:55:52,595 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config30 02:55:52,595 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config30: Creating local shard: member-1-shard-default-config30 02:55:52,604 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:55:52,604 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config30 02:55:52,605 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config30: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=non-existent, waitUntilReady=false] 02:55:52,605 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config30: No shard found for non-existent 02:55:52,706 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$p#771524951] 02:55:52,717 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config30 02:55:52,720 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config31 02:55:52,721 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config31 02:55:52,731 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config31: onAddShardReplica: AddShardReplica[ShardName=model-inventory] 02:55:52,731 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config31: No module configuration exists for shard model-inventory 02:55:52,832 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$q#1961440264] 02:55:52,843 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Stopping ShardManager shard-manager-config31 02:55:52,845 PM [main] [INFO] ShardManagerTest - testAddShardReplicaWithPreExistingReplicaInRemoteShardLeader starting 02:55:52,847 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config32 02:55:52,848 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config32 02:55:52,848 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config32: Creating local shard: member-1-shard-default-config32 02:55:52,857 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:55:52,857 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config32 02:55:52,857 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config32: Initializing shard [default] 02:55:52,857 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardInformation - Shard member-1-shard-default-config32 is initialized 02:55:52,858 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - shard-manager-config32: Received MemberUp: memberName: MemberName{name=leader-member}, address: akka://test 02:55:52,858 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardInformation - updatePeerAddress for peer leader-member-shard-default-config32 with address akka://test/user/shardmanager-config32/leader-member-shard-default-config32 02:55:52,858 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer leader-member-shard-default-config32 with address akka://test/user/shardmanager-config32/leader-member-shard-default-config32 to akka://test/user/member-1-shard-default-config 02:55:52,858 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - shard-manager-config32: Received role changed for member-1-shard-default-config32 from Candidate to Follower 02:55:52,858 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - shard-manager-config32: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config32, leaderId=leader-member-shard-default-config32, leaderPayloadVersion=13] 02:55:52,858 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - shard-manager-config32: All Shards are ready - data store config32 is ready 02:55:52,858 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config32: onAddShardReplica: AddShardReplica[ShardName=default] 02:55:52,858 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config32: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 02:55:52,858 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config32: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://test/user/shardmanager-config32/leader-member-shard-default-config32, primaryVersion=13] 02:55:52,859 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config32: Sending AddServer message to peer akka://test/user/shardmanager-config32/leader-member-shard-default-config32 for shard member-1-shard-default-config32 02:55:52,859 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config32: Received AddServerReply{status=ALREADY_EXISTS} for shard default from leader akka://test/user/shardmanager-config32/leader-member-shard-default-config32 02:55:52,859 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config32: Local shard default already exists 02:55:52,908 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config32: findLocalShard : default 02:55:52,908 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config32: onAddShardReplica: AddShardReplica[ShardName=default] 02:55:52,908 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config32: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 02:55:52,909 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config32: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://test/user/shardmanager-config32/leader-member-shard-default-config32, primaryVersion=13] 02:55:52,909 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config32: Sending AddServer message to peer akka://test/user/shardmanager-config32/leader-member-shard-default-config32 for shard member-1-shard-default-config32 02:55:52,909 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config32: Received AddServerReply{status=ALREADY_EXISTS} for shard default from leader akka://test/user/shardmanager-config32/leader-member-shard-default-config32 02:55:52,909 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config32: Local shard default already exists 02:55:52,910 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-default-config32 02:55:52,910 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config32: onAddShardReplica: AddShardReplica[ShardName=default] 02:55:52,910 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config32: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 02:55:52,911 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config32: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://test/user/shardmanager-config32/leader-member-shard-default-config32, primaryVersion=13] 02:55:52,911 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config32: Sending AddServer message to peer akka://test/user/shardmanager-config32/leader-member-shard-default-config32 for shard member-1-shard-default-config32 02:55:53,23 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config32: AddServer request to akka://test/user/shardmanager-config32/leader-member-shard-default-config32 for default failed akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://test/), Path(/user/shardmanager-config32/leader-member-shard-default-config32)]] after [100 ms]. Message of type [org.opendaylight.controller.cluster.raft.messages.AddServer]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 02:55:53,24 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config32: findLocalShard : default 02:55:53,24 PM [main] [INFO] ShardManagerTest - testAddShardReplicaWithPreExistingReplicaInRemoteShardLeader ending 02:55:53,126 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/shardmanager-config32] 02:55:53,137 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Stopping ShardManager shard-manager-config32 02:55:53,139 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForRemoteShard starting 02:55:53,153 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 02:55:53,162 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [1329204084513646840] 02:55:53,162 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 02:55:53,165 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 02:55:53,165 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 02:55:53,165 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:55:53,165 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:55:53,173 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 02:55:53,173 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 02:55:53,174 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 02:55:53,180 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config33] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:55:53,186 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Starting ShardManager shard-manager-config33 02:55:53,187 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 02:55:53,189 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config33 02:55:53,189 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config33: Creating local shard: member-1-shard-default-config33 02:55:53,190 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config33: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:55:53,190 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-default-config33 02:55:53,203 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [2173843896328957524] 02:55:53,203 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 02:55:53,215 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:55:53,215 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 02:55:53,219 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config33] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:55:53,221 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config33 02:55:53,228 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:55:53,228 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config33 02:55:53,228 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-default-config33 02:55:53,228 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-default-config33 to akka://test/user/member-1-shard-default-config 02:55:53,228 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config33: updated peer member-1-shard-default-config33 on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-default-config33 on shard member-1-shard-default-config33 whose actor address is Actor[akka://test/user/member-1-shard-default-config#-559888117] 02:55:53,230 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:55:53,230 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:55:53,231 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config33 02:55:53,231 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config33: Creating local shard: member-2-shard-default-config33 02:55:53,232 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config33: Creating local shard: member-2-shard-astronauts-config33 02:55:53,232 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:55:53,232 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-default-config33 02:55:53,232 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-astronauts-config33 02:55:53,232 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config33: Initializing shard [astronauts] 02:55:53,232 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-astronauts-config33 is initialized 02:55:53,232 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config33: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-astronauts-config33, leaderId=member-2-shard-astronauts-config33, leaderPayloadVersion=12] 02:55:53,232 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config33: Received role changed for member-2-shard-astronauts-config33 from Candidate to Leader 02:55:53,337 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 02:55:53,341 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 02:55:53,341 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config33: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:55:53,341 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-default-config33 02:55:53,341 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-default-config33 to akka://cluster-test/user/member-2-shard-astronauts-config 02:55:53,341 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-astronauts-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-astronauts-config33 02:55:53,341 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-astronauts-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-astronauts-config33 to akka://cluster-test/user/member-2-shard-astronauts-config 02:55:54,184 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 02:55:54,185 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardManager - shard-manager-config33: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:55:54,185 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config33 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33/member-2-shard-default-config33 02:55:54,185 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config33 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33/member-2-shard-default-config33 to akka://test/user/member-1-shard-default-config 02:55:54,185 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config33: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=astronauts, waitUntilReady=false] 02:55:54,185 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config33: findPrimary for astronauts forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33, visitedAddresses: [akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33] 02:55:54,192 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config33: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=astronauts, waitUntilReady=false] 02:55:54,192 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config33: Found primary for astronauts: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/member-2-shard-astronauts-config#1041743125, primaryVersion=12] 02:55:54,195 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForRemoteShard ending 02:55:54,248 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config33: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:55:54,248 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config33 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33/member-2-shard-default-config33 02:55:54,248 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config33 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33/member-2-shard-default-config33 to akka://cluster-test/user/member-2-shard-astronauts-config 02:55:54,248 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-astronauts-config33 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33/member-2-shard-astronauts-config33 02:55:54,248 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-astronauts-config33 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33/member-2-shard-astronauts-config33 to akka://cluster-test/user/member-2-shard-astronauts-config 02:55:54,297 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-config33 02:55:54,297 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:55:54,297 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:55:54,307 PM [cluster-test-akka.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:55:54,307 PM [cluster-test-akka.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:55:54,311 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 02:55:54,325 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config33 02:55:54,326 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:55:54,326 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:55:55,340 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 14:55:55.342] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:55:55,352 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Starting ShardManager shard-manager-config34 02:55:55,353 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config34 02:55:55,353 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config34: Creating local shard: member-1-shard-default-config34 02:55:55,363 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config34: findLocalShard : default 02:55:55,363 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config34: Returning NotInitializedException for shard default 02:55:55,464 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$r#-257782713] 02:55:55,475 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Stopping ShardManager shard-manager-config34 02:55:55,479 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Starting ShardManager shard-manager-config35 02:55:55,480 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config35 02:55:55,480 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config35: Creating local shard: member-1-shard-default-config35 02:55:55,488 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:55:55,488 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config35 02:55:55,488 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config35: Initializing shard [default] 02:55:55,488 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardInformation - Shard member-1-shard-default-config35 is initialized 02:55:55,488 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - shard-manager-config35: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config35, leaderId=member-1-shard-default-config35, leaderPayloadVersion=13] 02:55:55,488 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - shard-manager-config35: Received role changed for member-1-shard-default-config35 from Candidate to Leader 02:55:55,488 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - shard-manager-config35: All Shards are ready - data store config35 is ready 02:55:55,488 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config35: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=default, meberVotingStatusMap={member-2=true}] 02:55:55,488 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config35: findLocalShard : default 02:55:55,489 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config35: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-default-config35=true}, serversVisited=[]] to local shard akka://test/user/member-1-shard-default-config35 02:55:55,489 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config35: Received ServerChangeReply{status=OK} from local shard akka://test/user/member-1-shard-default-config35 02:55:55,489 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config35: ChangeServersVotingStatus succeeded for shard default 02:55:55,640 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-default-config35#2012091057] 02:55:55,653 PM [main] [INFO] ShardManagerTest - testShutDown starting 02:55:55,656 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Starting ShardManager shard-manager-config36 02:55:55,664 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config36 02:55:55,664 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config36: Creating local shard: member-1-shard-shard2-config36 02:55:55,664 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config36: Creating local shard: member-1-shard-shard1-config36 02:55:55,666 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:55:55,666 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - Creating Shard member-1-shard-shard2-config36 02:55:55,666 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - Creating Shard member-1-shard-shard1-config36 02:55:55,666 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config36: Initializing shard [shard1] 02:55:55,666 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardInformation - Shard member-1-shard-shard1-config36 is initialized 02:55:55,666 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config36: Initializing shard [shard2] 02:55:55,666 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardInformation - Shard member-1-shard-shard2-config36 is initialized 02:55:55,667 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config36: Issuing gracefulStop to shard member-1-shard-shard2-config36 02:55:55,667 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config36: Issuing gracefulStop to shard member-1-shard-shard1-config36 02:55:55,668 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Shutting down ShardManager shard-manager-config36 - waiting on 2 shards 02:55:56,218 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-shard1-config36#-1698859831] 02:55:56,219 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-shard2-config36#-339440217] 02:55:56,219 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config36: All shards shutdown - sending PoisonPill to self 02:55:56,220 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-config36 02:55:56,220 PM [main] [INFO] ShardManagerTest - testShutDown ending 02:55:56,322 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$t#-317286865] 02:55:56,334 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForNonLocalLeaderShardBeforeMemberUp starting 02:55:56,336 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config37 02:55:56,341 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config37 02:55:56,341 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config37: Creating local shard: member-1-shard-default-config37 02:55:56,347 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:55:56,347 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config37 02:55:56,347 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config37: Initializing shard [default] 02:55:56,347 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardInformation - Shard member-1-shard-default-config37 is initialized 02:55:56,347 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - shard-manager-config37: Received role changed for member-1-shard-default-config37 from Candidate to Follower 02:55:56,347 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManagerTest$TestShardManager - Received unhandled message LeaderStateChanged [memberId=member-1-shard-default-config37, leaderId=member-2-shard-default-config37, leaderPayloadVersion=13] 02:55:56,348 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config37: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 02:55:56,348 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config37: Returning NoShardLeaderException for shard default 02:55:56,348 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForNonLocalLeaderShardBeforeMemberUp ending 02:55:56,450 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$u#-71891444] 02:55:56,461 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config37 02:55:56,462 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithNoRoleShard starting 02:55:56,465 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config38 02:55:56,466 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config38 02:55:56,466 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config38: Creating local shard: member-1-shard-default-config38 02:55:56,475 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:55:56,475 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config38 02:55:56,475 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config38: Initializing shard [default] 02:55:56,475 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config38 is initialized 02:55:56,475 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config38: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 02:55:56,475 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config38: Scheduling 1200 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config38, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 02:55:57,693 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config38: Received ShardNotInitializedTimeout message for shard default 02:55:57,693 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config38: Returning NoShardLeaderException for shard default 02:55:57,694 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithNoRoleShard ending 02:55:57,795 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$v#1634228899] 02:55:57,806 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config38 02:55:57,808 PM [main] [INFO] ShardManagerTest - testShardPersistenceWithRestoredData starting 02:55:57,814 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config39 02:55:57,815 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config39: onSnapshotOffer: ShardManagerSnapshot [ShardList = [default, astronauts] ] 02:55:57,815 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config39: adding shard astronauts 02:55:57,816 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config39: removing shard people 02:55:57,816 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config39 02:55:57,816 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config39: Creating local shard: member-1-shard-default-config39 02:55:57,817 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config39: Creating local shard: member-1-shard-astronauts-config39 02:55:57,921 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config39: findLocalShard : people 02:55:57,921 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config39: Local shard people not found - shards present: [default, astronauts] 02:55:57,921 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config39: findLocalShard : default 02:55:57,922 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config39: Returning NotInitializedException for shard default 02:55:57,922 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config39: findLocalShard : astronauts 02:55:57,922 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config39: Returning NotInitializedException for shard astronauts 02:55:57,922 PM [main] [INFO] ShardManagerTest - testShardPersistenceWithRestoredData ending 02:55:58,24 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$i] 02:55:58,34 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config39 02:55:58,36 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForNonLocalLeaderShard starting 02:55:58,38 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config40 02:55:58,41 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config40 02:55:58,41 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config40: Creating local shard: member-1-shard-default-config40 02:55:58,49 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:55:58,49 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config40 02:55:58,49 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config40: Initializing shard [default] 02:55:58,49 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config40 is initialized 02:55:58,49 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config40: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://test 02:55:58,49 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config40 with address akka://test/user/shardmanager-config40/member-2-shard-default-config40 02:55:58,49 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config40 with address akka://test/user/shardmanager-config40/member-2-shard-default-config40 to akka://test/user/member-1-shard-default-config 02:55:58,49 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config40: Received role changed for member-1-shard-default-config40 from Candidate to Follower 02:55:58,49 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config40: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config40, leaderId=member-2-shard-default-config40, leaderPayloadVersion=12] 02:55:58,49 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config40: All Shards are ready - data store config40 is ready 02:55:58,49 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config40: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 02:55:58,49 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config40: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://test/user/shardmanager-config40/member-2-shard-default-config40, primaryVersion=12] 02:55:58,50 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForNonLocalLeaderShard ending 02:55:58,151 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$w#-576970451] 02:55:58,162 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config40 02:55:58,164 PM [main] [INFO] ShardManagerTest - testShardAvailabilityOnChangeOfMemberReachability starting 02:55:58,171 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 02:55:58,184 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [5113231376070835191] 02:55:58,184 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 02:55:58,186 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 02:55:58,186 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 02:55:58,189 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:55:58,189 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:55:58,202 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config41 02:55:58,207 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 02:55:58,207 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 02:55:58,207 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 02:55:58,207 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config41] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:55:58,208 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] ShardManager - Recovery complete : shard-manager-config41 02:55:58,208 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config41: Creating local shard: member-1-shard-default-config41 02:55:58,208 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] ShardManager - shard-manager-config41: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:55:58,208 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-1-shard-default-config41 02:55:58,210 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 02:55:58,236 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [1562927349386903920] 02:55:58,236 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 02:55:58,237 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:55:58,237 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 02:55:58,245 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:55:58,245 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:55:58,254 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config41] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:55:58,255 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Starting ShardManager shard-manager-config41 02:55:58,259 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:55:58,259 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config41 02:55:58,259 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-1-shard-default-config41 02:55:58,259 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-1-shard-default-config41 to akka://cluster-test/user/member-1-shard-default-config 02:55:58,260 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config41: updated peer member-1-shard-default-config41 on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-1-shard-default-config41 on shard member-1-shard-default-config41 whose actor address is Actor[akka://cluster-test/user/member-1-shard-default-config#1910780041] 02:55:58,260 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config41: Initializing shard [default] 02:55:58,260 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-default-config41 is initialized 02:55:58,260 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config41: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config41, leaderId=member-2-shard-default-config41, leaderPayloadVersion=13] 02:55:58,260 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config41: Received role changed for member-1-shard-default-config41 from Candidate to Follower 02:55:58,261 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config41 02:55:58,261 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config41: Creating local shard: member-2-shard-default-config41 02:55:58,261 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:55:58,261 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-default-config41 02:55:58,261 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config41: Initializing shard [default] 02:55:58,261 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-default-config41 is initialized 02:55:58,261 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config41: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-config41, leaderId=member-2-shard-default-config41, leaderPayloadVersion=13] 02:55:58,261 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config41: Received role changed for member-2-shard-default-config41 from Candidate to Leader 02:55:58,261 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config41: All Shards are ready - data store config41 is ready 02:55:58,313 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 02:55:58,316 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 02:55:58,316 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config41: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:55:58,316 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-1-shard-default-config41 02:55:58,316 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-1-shard-default-config41 to akka://cluster-test/user/member-2-shard-default-config 02:55:58,316 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config41: All Shards are ready - data store config41 is ready 02:55:59,200 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 02:55:59,200 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config41: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:55:59,200 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config41/member-2-shard-default-config41 02:55:59,200 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config41/member-2-shard-default-config41 to akka://cluster-test/user/member-1-shard-default-config 02:55:59,200 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config41: All Shards are ready - data store config41 is ready 02:55:59,201 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config41: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 02:55:59,201 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config41: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config41/member-2-shard-default-config41, primaryVersion=13] 02:55:59,201 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 02:55:59,201 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Marking Leader member-2-shard-default-config41 as unavailable. 02:55:59,202 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config41: Received MemberRemoved: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 02:55:59,202 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config41: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 02:55:59,202 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config41: Scheduling 1200 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config41, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=member-2-shard-default-config41, activeMember=true] 02:55:59,261 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config41: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:55:59,261 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config41/member-2-shard-default-config41 02:55:59,261 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config41/member-2-shard-default-config41 to akka://cluster-test/user/member-2-shard-default-config 02:55:59,261 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config41: All Shards are ready - data store config41 is ready 02:56:00,418 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config41: Received ShardNotInitializedTimeout message for shard default 02:56:00,418 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config41: Returning NoShardLeaderException for shard default 02:56:00,419 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Received ReachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 02:56:00,419 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41 02:56:00,419 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41 to akka://cluster-test/user/member-1-shard-default-config 02:56:00,419 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Marking Leader member-2-shard-default-config41 as available. 02:56:00,419 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config41: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 02:56:00,419 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config41: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41, primaryVersion=13] 02:56:00,420 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config41: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 02:56:00,420 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41 02:56:00,420 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41 to akka://cluster-test/user/member-1-shard-default-config 02:56:00,420 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config41: All Shards are ready - data store config41 is ready 02:56:00,420 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 02:56:00,420 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Marking Leader member-2-shard-default-config41 as unavailable. 02:56:00,422 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config41: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 02:56:00,422 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config41: Scheduling 1200 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config41, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=member-2-shard-default-config41, activeMember=true] 02:56:00,422 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Received ReachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 02:56:00,422 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41 02:56:00,422 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41 to akka://cluster-test/user/member-1-shard-default-config 02:56:00,422 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-default-config41 is initialized - notifying 1 OnShardInitialized callbacks 02:56:00,422 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Marking Leader member-2-shard-default-config41 as available. 02:56:00,422 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-default-config41 is ready - notifying 1 OnShardInitialized callbacks 02:56:00,422 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config41: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41, primaryVersion=13] 02:56:00,422 PM [main] [INFO] ShardManagerTest - testShardAvailabilityOnChangeOfMemberReachability ending 02:56:00,524 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config41 02:56:00,525 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:56:00,525 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:56:00,531 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:56:00,531 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:56:00,532 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 02:56:00,550 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config41 02:56:00,551 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:56:00,551 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:56:01,568 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 14:56:01.570] [cluster-test-akka.actor.internal-dispatcher-5] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:56:01,585 PM [main] [INFO] ShardManagerTest - testAddShardReplica starting 02:56:01,592 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 02:56:01,602 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-5755993099028585578] 02:56:01,602 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 02:56:01,605 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 02:56:01,606 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 02:56:01,640 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:56:01,640 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:56:01,640 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 02:56:01,640 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 02:56:01,642 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 02:56:01,642 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config42] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:56:01,644 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shardmanager-config42 02:56:01,646 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 02:56:01,646 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shardmanager-config42 02:56:01,647 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shardmanager-config42: Creating local shard: member-1-shard-default-config42 02:56:01,647 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shardmanager-config42: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:56:01,647 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-default-config42 02:56:01,654 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-4547147880661731128] 02:56:01,654 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 02:56:01,654 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:56:01,654 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 02:56:01,660 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:56:01,661 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:56:01,673 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shardmanager-config42 02:56:01,673 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config42] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:56:01,674 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:56:01,674 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config42 02:56:01,674 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-default-config42 02:56:01,674 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-default-config42 to akka://cluster-test/user/member-1-shard-default-config 02:56:01,674 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shardmanager-config42: updated peer member-1-shard-default-config42 on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-default-config42 on shard member-1-shard-default-config42 whose actor address is Actor[akka://cluster-test/user/member-1-shard-default-config#-998044964] 02:56:01,684 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shardmanager-config42 02:56:01,684 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shardmanager-config42: Creating local shard: member-2-shard-default-config42 02:56:01,685 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shardmanager-config42: Creating local shard: member-2-shard-astronauts-config42 02:56:01,685 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:56:01,685 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-2-shard-default-config42 02:56:01,685 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-2-shard-astronauts-config42 02:56:01,685 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shardmanager-config42: Initializing shard [astronauts] 02:56:01,685 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Shard member-2-shard-astronauts-config42 is initialized 02:56:01,685 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shardmanager-config42: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-astronauts-config42, leaderId=member-2-shard-astronauts-config42, leaderPayloadVersion=12] 02:56:01,685 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shardmanager-config42: Received role changed for member-2-shard-astronauts-config42 from Candidate to Leader 02:56:01,726 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 02:56:01,731 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 02:56:01,732 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shardmanager-config42: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:56:01,732 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-default-config42 02:56:01,732 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-default-config42 to akka://cluster-test/user/member-2-shard-astronauts-config 02:56:01,732 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-astronauts-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-astronauts-config42 02:56:01,732 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-astronauts-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-astronauts-config42 to akka://cluster-test/user/member-2-shard-astronauts-config 02:56:02,650 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 02:56:02,651 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shardmanager-config42: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:56:02,651 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config42 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42/member-2-shard-default-config42 02:56:02,651 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config42 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42/member-2-shard-default-config42 to akka://cluster-test/user/member-1-shard-default-config 02:56:02,654 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shardmanager-config42: onAddShardReplica: AddShardReplica[ShardName=astronauts] 02:56:02,655 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shardmanager-config42: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=astronauts, waitUntilReady=true] 02:56:02,655 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shardmanager-config42: findPrimary for astronauts forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42, visitedAddresses: [akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42] 02:56:02,661 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shardmanager-config42: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=astronauts, waitUntilReady=true] 02:56:02,661 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shardmanager-config42: Found primary for astronauts: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/member-2-shard-astronauts-config, primaryVersion=12] 02:56:02,664 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shardmanager-config42: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2559/user/member-2-shard-astronauts-config for shard member-1-shard-astronauts-config42 02:56:02,677 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shardmanager-config42: Received AddServerReply{status=OK, leaderHint=member-2-shard-astronauts-config42} for shard astronauts from leader akka://cluster-test@127.0.0.1:2559/user/member-2-shard-astronauts-config 02:56:02,677 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shardmanager-config42: Leader shard successfully added the replica shard astronauts 02:56:02,677 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-astronauts-config42 02:56:02,677 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shardmanager-config42: persisting the shard list [default, astronauts] 02:56:02,678 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shardmanager-config42 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 02:56:02,679 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shardmanager-config42: Successfully deleted prior snapshots 02:56:02,681 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shardmanager-config42: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:56:02,681 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config42 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42/member-2-shard-default-config42 02:56:02,681 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config42 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42/member-2-shard-default-config42 to akka://cluster-test/user/member-2-shard-astronauts-config 02:56:02,681 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-astronauts-config42 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42/member-2-shard-astronauts-config42 02:56:02,681 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-astronauts-config42 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42/member-2-shard-astronauts-config42 to akka://cluster-test/user/member-2-shard-astronauts-config 02:56:02,706 PM [main] [INFO] ShardManagerTest - testAddShardReplica ending 02:56:02,808 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shardmanager-config42 02:56:02,810 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:56:02,810 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/11/2024 14:56:02.816] [cluster-test-akka.remote.default-remote-dispatcher-7] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-601)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/11/2024 14:56:02.816] [cluster-test-akka.remote.default-remote-dispatcher-8] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-601)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:56:02,830 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shardmanager-config42 02:56:02,831 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:56:02,831 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:56:03,844 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 14:56:03.846] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:56:03,868 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config43 02:56:03,870 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config43 02:56:03,870 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config43: Creating local shard: member-1-shard-default-config43 02:56:03,877 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:56:03,877 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config43 02:56:03,877 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config43: Initializing shard [default] 02:56:03,877 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-default-config43 is initialized 02:56:03,877 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config43: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config43, leaderId=member-1-shard-default-config43, leaderPayloadVersion=13] 02:56:03,878 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config43: Received role changed for member-1-shard-default-config43 from Candidate to Leader 02:56:03,878 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config43: All Shards are ready - data store config43 is ready 02:56:03,878 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config43: onRemoveShardReplica: RemoveShardReplica [shardName=default, memberName=MemberName{name=member-1}] 02:56:03,878 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config43: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 02:56:03,878 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config43: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://test@127.0.0.1:2565/user/member-1-shard-default-config43#-1401079489, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$nA3uBRwe@20608148] 02:56:03,878 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config43: Sending RemoveServer message to peer akka://test@127.0.0.1:2565/user/member-1-shard-default-config43#-1401079489 for shard member-1-shard-default-config43 02:56:03,878 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config43: Received RemoveServerReply{status=OK} for shard default 02:56:03,878 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config43: Leader shard successfully removed the replica shard default 02:56:04,29 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-default-config43#-1401079489] 02:56:04,42 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithIsolatedLeaderShard starting 02:56:04,44 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config44 02:56:04,47 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config44 02:56:04,47 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config44: Creating local shard: member-1-shard-default-config44 02:56:04,55 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:56:04,55 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config44 02:56:04,55 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config44: Initializing shard [default] 02:56:04,55 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-default-config44 is initialized 02:56:04,55 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config44: Received role changed for member-1-shard-default-config44 from null to IsolatedLeader 02:56:04,55 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config44: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 02:56:04,55 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config44: Scheduling 1200 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config44, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=null, activeMember=true] 02:56:05,274 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config44: Received ShardNotInitializedTimeout message for shard default 02:56:05,274 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config44: Returning NoShardLeaderException for shard default 02:56:05,274 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithIsolatedLeaderShard ending 02:56:05,376 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$y#944536772] 02:56:05,387 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config44 02:56:05,391 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config45 02:56:05,392 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config45 02:56:05,392 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config45: Creating local shard: member-1-shard-default-config45 02:56:05,401 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:56:05,401 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config45 02:56:05,402 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config45: findLocalShard : non-existent 02:56:05,402 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config45: Local shard non-existent not found - shards present: [default] 02:56:05,504 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$z#854127299] 02:56:05,515 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config45 02:56:05,519 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config46 02:56:05,520 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config46 02:56:05,540 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:56:05,540 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config46: onRemoveShardReplica: RemoveShardReplica [shardName=astronauts, memberName=MemberName{name=member-2}] 02:56:05,541 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config46: Sending RemoveServer message to peer akka://test@127.0.0.1:2565/system/testProbe-595#-296507972 for shard member-2-shard-astronauts-config46 02:56:05,552 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config46: onRemoveShardReplica: RemoveShardReplica [shardName=astronauts, memberName=MemberName{name=member-3}] 02:56:05,552 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config46: A shard replica operation for astronauts is already in progress 02:56:05,663 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config47 02:56:05,664 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config47 02:56:05,664 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config47: Creating local shard: member-1-shard-default-config47 02:56:05,673 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config47: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 02:56:05,673 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config47: Returning NotInitializedException for shard default 02:56:05,775 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$A#-1369788848] 02:56:05,786 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config47 02:56:05,788 PM [main] [INFO] ShardManagerTest - testServerRemovedShardActorRunning starting 02:56:05,792 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config48 02:56:05,793 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config48 02:56:05,793 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config48: Creating local shard: member-1-shard-default-config48 02:56:05,793 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config48: Creating local shard: member-1-shard-people-config48 02:56:05,804 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:56:05,804 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config48 02:56:05,804 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config48 02:56:05,804 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config48: Initializing shard [default] 02:56:05,804 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-default-config48 is initialized 02:56:05,804 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config48: findLocalShard : people 02:56:05,804 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config48: Scheduling 600 ms timer to wait for shard ShardInformation [shardId=member-1-shard-people-config48, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 02:56:05,807 PM [test-shard-dispatcher-17] [INFO] Shard - Shard created : member-1-shard-people-config48, persistent : true 02:56:05,810 PM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$j/member-1-shard-people-config48/member-1-shard-people-config48-notifier#-1058848794 created and ready for shard:member-1-shard-people-config48 02:56:05,810 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$j/member-1-shard-people-config48/shard-member-1:snapshot-read#-1096107862] 02:56:05,811 PM [test-shard-dispatcher-17] [INFO] Shard - Starting recovery for member-1-shard-people-config48 with journal batch size 1 02:56:05,812 PM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config48: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 02:56:05,812 PM [test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config48: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:56:05,812 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config48: Initializing shard [people] 02:56:05,812 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-people-config48 is initialized 02:56:05,812 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-people-config48 is initialized - notifying 1 OnShardInitialized callbacks 02:56:05,813 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config48: findLocalShard : default 02:56:05,813 PM [test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config48: onLeadershipChange, isLeader: false, hasLeader : false 02:56:05,813 PM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config48: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config48 is no longer the leader 02:56:05,813 PM [test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config48 , received role change from null to Follower 02:56:05,813 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config48 : Sending Shutdown to Shard actor Actor[akka://test/user/member-1-shard-default-config48#1033191685] with 10000 ms timeout 02:56:05,813 PM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config48 , registered listener akka://test/user/$$j 02:56:05,816 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config48 : Local Shard replica for shard default has been removed 02:56:05,816 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config48: persisting the shard list [people] 02:56:05,816 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config48: Received role changed for member-1-shard-people-config48 from null to Follower 02:56:05,816 PM [main] [INFO] ShardManagerTest - testServerRemovedShardActorRunning ending 02:56:05,816 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config48 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 02:56:05,817 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config48: Successfully deleted prior snapshots 02:56:05,918 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-default-config48#1033191685] 02:56:05,929 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$j] 02:56:05,929 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config48 : Successfully shut down Shard actor Actor[akka://test/user/member-1-shard-default-config48#1033191685] 02:56:05,929 PM [test-shard-dispatcher-17] [INFO] Shard - Stopping Shard member-1-shard-people-config48 02:56:05,930 PM [test-shard-dispatcher-17] [DEBUG] MessageAssembler - member-1-shard-people-config48 (Follower): Closing 02:56:05,930 PM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config48: Aborting 0 pending queued transactions 02:56:05,930 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-config48 02:56:05,935 PM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config49 02:56:05,936 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config49 02:56:05,936 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config49: Creating local shard: member-1-shard-default-config49 02:56:05,936 PM [main] [INFO] ShardManager - shard-manager-config49: Received role changed for member-1-shard-default-config49 from null to Follower 02:56:05,937 PM [main] [INFO] ShardManager - shard-manager-config49: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config49, leaderId=member-2-shard-default-config49, leaderPayloadVersion=13] 02:56:05,947 PM [main] [INFO] ShardManager - shard-manager-config49: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://test 02:56:05,947 PM [main] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config49 with address akka://test/user/shardmanager-config49/member-2-shard-default-config49 02:56:05,947 PM [main] [INFO] ShardManager - shard-manager-config49: All Shards are ready - data store config49 is ready 02:56:06,49 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$k] 02:56:06,59 PM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config49 02:56:06,61 PM [main] [INFO] ShardManagerTest - testRestoreFromSnapshot starting 02:56:06,64 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config50 02:56:06,66 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config50 02:56:06,66 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config50: Restoring from ShardManagerSnapshot: ShardManagerSnapshot [ShardList = [shard1, shard2, astronauts] ] 02:56:06,66 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config50: onSnapshotOffer: ShardManagerSnapshot [ShardList = [shard1, shard2, astronauts] ] 02:56:06,66 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config50: adding shard shard1 02:56:06,66 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config50: adding shard shard2 02:56:06,66 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config50: adding shard astronauts 02:56:06,66 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config50: Creating local shard: member-1-shard-shard2-config50 02:56:06,66 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config50: Creating local shard: member-1-shard-astronauts-config50 02:56:06,66 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config50: Creating local shard: member-1-shard-shard1-config50 02:56:06,67 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:56:06,67 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-shard2-config50 02:56:06,67 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-astronauts-config50 02:56:06,68 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-shard1-config50 02:56:06,68 PM [test-shard-dispatcher-17] [INFO] Shard - Shard created : member-1-shard-shard2-config50, persistent : true 02:56:06,69 PM [test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-astronauts-config50, persistent : true 02:56:06,69 PM [test-shard-dispatcher-17] [INFO] Shard - Starting recovery for member-1-shard-shard2-config50 with journal batch size 1 02:56:06,69 PM [test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$l/member-1-shard-shard2-config50/member-1-shard-shard2-config50-notifier#-1151125328 created and ready for shard:member-1-shard-shard2-config50 02:56:06,69 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$l/member-1-shard-shard2-config50/shard-member-1:snapshot-read#-268326261] 02:56:06,69 PM [test-shard-dispatcher-17] [INFO] Shard - Shard created : member-1-shard-shard1-config50, persistent : true 02:56:06,70 PM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$l/member-1-shard-astronauts-config50/member-1-shard-astronauts-config50-notifier#-995232061 created and ready for shard:member-1-shard-astronauts-config50 02:56:06,70 PM [test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$l/member-1-shard-shard1-config50/member-1-shard-shard1-config50-notifier#241158882 created and ready for shard:member-1-shard-shard1-config50 02:56:06,70 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$l/member-1-shard-shard1-config50/shard-member-1:snapshot-read#224364712] 02:56:06,70 PM [test-shard-dispatcher-17] [INFO] Shard - Starting recovery for member-1-shard-shard1-config50 with journal batch size 1 02:56:06,71 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$l/member-1-shard-astronauts-config50/shard-member-1:snapshot-read#1547163248] 02:56:06,72 PM [test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-astronauts-config50 with journal batch size 1 02:56:06,73 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config50: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 02:56:06,73 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard1-config50: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 02:56:06,73 PM [test-shard-dispatcher-19] [INFO] Shard - member-1-shard-shard2-config50: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:56:06,73 PM [test-shard-dispatcher-20] [INFO] Shard - member-1-shard-shard1-config50: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:56:06,73 PM [test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard2-config50: onLeadershipChange, isLeader: false, hasLeader : false 02:56:06,73 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config50: onStateChanged: Closing all transaction chains because shard member-1-shard-shard2-config50 is no longer the leader 02:56:06,73 PM [test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard1-config50: onLeadershipChange, isLeader: false, hasLeader : false 02:56:06,73 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard1-config50: onStateChanged: Closing all transaction chains because shard member-1-shard-shard1-config50 is no longer the leader 02:56:06,73 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config50: Initializing shard [shard2] 02:56:06,73 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-shard2-config50 is initialized 02:56:06,73 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config50: Initializing shard [shard1] 02:56:06,73 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-shard1-config50 is initialized 02:56:06,73 PM [test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config50 , received role change from null to Follower 02:56:06,73 PM [test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config50 , received role change from null to Follower 02:56:06,74 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config50 (Follower): Received TimeoutNow - switching to Candidate 02:56:06,74 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard1-config50 (Follower): Received TimeoutNow - switching to Candidate 02:56:06,76 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard1-config50 (Candidate): Election: Candidate has following voting peers: [] 02:56:06,76 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 02:56:06,76 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard1-config50: Set currentTerm=1, votedFor=member-1-shard-shard1-config50 02:56:06,76 PM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-astronauts-config50: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:56:06,76 PM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config50: onLeadershipChange, isLeader: false, hasLeader : false 02:56:06,76 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50: onStateChanged: Closing all transaction chains because shard member-1-shard-astronauts-config50 is no longer the leader 02:56:06,77 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50 (Follower): Received TimeoutNow - switching to Candidate 02:56:06,77 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50 (Candidate): Election: Candidate has following voting peers: [] 02:56:06,77 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50: Set currentTerm=1, votedFor=member-1-shard-astronauts-config50 02:56:06,77 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config50: Initializing shard [astronauts] 02:56:06,77 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardInformation - Shard member-1-shard-astronauts-config50 is initialized 02:56:06,77 PM [test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config50 , received role change from null to Follower 02:56:06,77 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config50 (Candidate): Election: Candidate has following voting peers: [] 02:56:06,77 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config50: Set currentTerm=1, votedFor=member-1-shard-shard2-config50 02:56:06,77 PM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config50: findLocalShard : shard1 02:56:06,77 PM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config50: findLocalShard : shard2 02:56:06,78 PM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config50: findLocalShard : astronauts 02:56:06,79 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config50: onGetSnapshot 02:56:06,80 PM [test-shard-dispatcher-19] [INFO] Shard - member-1-shard-shard2-config50 (Candidate): Starting new election term 1 02:56:06,80 PM [test-shard-dispatcher-19] [INFO] Shard - member-1-shard-shard2-config50 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:56:06,80 PM [test-shard-dispatcher-19] [DEBUG] MessageAssembler - member-1-shard-shard2-config50 (Follower): Closing 02:56:06,80 PM [test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard2-config50: onLeadershipChange, isLeader: false, hasLeader : false 02:56:06,80 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config50: onStateChanged: Closing all transaction chains because shard member-1-shard-shard2-config50 is no longer the leader 02:56:06,80 PM [test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config50 , received role change from Follower to Candidate 02:56:06,82 PM [test-shard-dispatcher-20] [INFO] Shard - member-1-shard-shard1-config50 (Candidate): Starting new election term 1 02:56:06,82 PM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config50: Expecting 3 shard snapshot replies 02:56:06,82 PM [test-shard-dispatcher-20] [INFO] Shard - member-1-shard-shard1-config50 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:56:06,82 PM [test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-1-shard-shard1-config50 (Follower): Closing 02:56:06,82 PM [test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard1-config50: onLeadershipChange, isLeader: false, hasLeader : false 02:56:06,82 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard1-config50: onStateChanged: Closing all transaction chains because shard member-1-shard-shard1-config50 is no longer the leader 02:56:06,82 PM [test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config50 , received role change from Follower to Candidate 02:56:06,86 PM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-astronauts-config50 (Candidate): Starting new election term 1 02:56:06,86 PM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-astronauts-config50 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:56:06,86 PM [test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-1-shard-astronauts-config50 (Follower): Closing 02:56:06,86 PM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config50: onLeadershipChange, isLeader: false, hasLeader : false 02:56:06,86 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50: onStateChanged: Closing all transaction chains because shard member-1-shard-astronauts-config50 is no longer the leader 02:56:06,86 PM [test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config50 , received role change from Follower to Candidate 02:56:06,92 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config50 (Candidate): Received ElectionTimeout 02:56:06,95 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config50 (Candidate): Received ElectionTimeout 02:56:06,97 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-astronauts-config50 (Candidate): Received ElectionTimeout 02:56:06,99 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config50 (Leader): Election: Leader has following peers: [] 02:56:06,99 PM [test-shard-dispatcher-19] [INFO] Shard - member-1-shard-shard2-config50 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 02:56:06,99 PM [test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard2-config50: onLeadershipChange, isLeader: true, hasLeader : true 02:56:06,101 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config50: became leader with frontend state for [] 02:56:06,101 PM [test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config50 , received role change from Candidate to Leader 02:56:06,103 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-astronauts-config50 (Leader): Election: Leader has following peers: [] 02:56:06,103 PM [test-shard-dispatcher-20] [INFO] Shard - member-1-shard-astronauts-config50 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 02:56:06,103 PM [test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config50: onLeadershipChange, isLeader: true, hasLeader : true 02:56:06,103 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-astronauts-config50: became leader with frontend state for [] 02:56:06,103 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config50 (Leader): Election: Leader has following peers: [] 02:56:06,103 PM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-shard1-config50 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 02:56:06,103 PM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard1-config50: onLeadershipChange, isLeader: true, hasLeader : true 02:56:06,103 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config50: became leader with frontend state for [] 02:56:06,104 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config50: onGetSnapshot 02:56:06,104 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config50: onGetSnapshot 02:56:06,104 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config50: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 02:56:06,104 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config50: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 02:56:06,104 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-astronauts-config50: onGetSnapshot 02:56:06,104 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-astronauts-config50: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 02:56:06,104 PM [test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config50 , received role change from Candidate to Leader 02:56:06,104 PM [test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config50 , registered listener akka://test/user/$$l 02:56:06,105 PM [test-akka.actor.default-dispatcher-21] [INFO] ShardManager - shard-manager-config50: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-shard1-config50, leaderId=member-1-shard-shard1-config50, leaderPayloadVersion=13] 02:56:06,105 PM [test-akka.actor.default-dispatcher-21] [INFO] ShardManager - shard-manager-config50: Received role changed for member-1-shard-shard1-config50 from Candidate to Leader 02:56:06,105 PM [test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config50 , registered listener akka://test/user/$$l 02:56:06,105 PM [test-akka.actor.default-dispatcher-21] [INFO] ShardManager - shard-manager-config50: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-shard2-config50, leaderId=member-1-shard-shard2-config50, leaderPayloadVersion=13] 02:56:06,105 PM [test-akka.actor.default-dispatcher-21] [INFO] ShardManager - shard-manager-config50: Received role changed for member-1-shard-shard2-config50 from Candidate to Leader 02:56:06,105 PM [test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config50 , received role change from Candidate to Leader 02:56:06,105 PM [test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config50 , registered listener akka://test/user/$$l 02:56:06,105 PM [test-akka.actor.default-dispatcher-21] [INFO] ShardManager - shard-manager-config50: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-astronauts-config50, leaderId=member-1-shard-astronauts-config50, leaderPayloadVersion=13] 02:56:06,105 PM [test-akka.actor.default-dispatcher-21] [INFO] ShardManager - shard-manager-config50: Received role changed for member-1-shard-astronauts-config50 from Candidate to Leader 02:56:06,105 PM [test-akka.actor.default-dispatcher-21] [INFO] ShardManager - shard-manager-config50: All Shards are ready - data store config50 is ready 02:56:06,109 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config50: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 02:56:06,109 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-astronauts-config50: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 02:56:06,109 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config50: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 02:56:06,115 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config50: Received GetSnapshotReply [id=member-1-shard-shard2-config50, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@8cff1e8, electionTerm=1, electionVotedFor=member-1-shard-shard2-config50, ServerConfigPayload=null]] 02:56:06,115 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config50: Received GetSnapshotReply [id=member-1-shard-astronauts-config50, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@667b9aee, electionTerm=1, electionVotedFor=member-1-shard-astronauts-config50, ServerConfigPayload=null]] 02:56:06,115 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config50: Received GetSnapshotReply [id=member-1-shard-shard1-config50, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@114c7778, electionTerm=1, electionVotedFor=member-1-shard-shard1-config50, ServerConfigPayload=null]] 02:56:06,115 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config50: All shard snapshots received 02:56:06,116 PM [main] [INFO] ShardManagerTest - testRestoreFromSnapshot ending 02:56:06,217 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$l] 02:56:06,229 PM [test-shard-dispatcher-20] [INFO] Shard - Stopping Shard member-1-shard-astronauts-config50 02:56:06,229 PM [test-shard-dispatcher-20] [DEBUG] MessageSlicer - member-1-shard-astronauts-config50 (Leader)_slicer-id-215: Closing 02:56:06,229 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-astronauts-config50: Aborting 0 pending queued transactions 02:56:06,229 PM [test-shard-dispatcher-20] [INFO] Shard - Stopping Shard member-1-shard-shard1-config50 02:56:06,229 PM [test-shard-dispatcher-20] [DEBUG] MessageSlicer - member-1-shard-shard1-config50 (Leader)_slicer-id-216: Closing 02:56:06,229 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard1-config50: Aborting 0 pending queued transactions 02:56:06,229 PM [test-shard-dispatcher-20] [INFO] Shard - Stopping Shard member-1-shard-shard2-config50 02:56:06,229 PM [test-shard-dispatcher-20] [DEBUG] MessageSlicer - member-1-shard-shard2-config50 (Leader)_slicer-id-214: Closing 02:56:06,229 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard2-config50: Aborting 0 pending queued transactions 02:56:06,230 PM [test-akka.actor.default-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-config50 02:56:06,231 PM [main] [INFO] ShardManagerTest - testOnCreateShardWithNoInitialSchemaContext starting 02:56:06,234 PM [test-akka.actor.default-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-config51 02:56:06,238 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config51 02:56:06,244 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config51: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=foo-ns, moduleName=foo-module, shardName=foo, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@599abaf6] 02:56:06,245 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config51 doCreateShard: shardId: member-1-shard-foo-config51, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: true 02:56:06,245 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:56:06,245 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-foo-config51 02:56:06,246 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config51: findLocalShard : foo 02:56:06,246 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config51: Scheduling 600 ms timer to wait for shard ShardInformation [shardId=member-1-shard-foo-config51, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 02:56:06,246 PM [test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-foo-config51, persistent : true 02:56:06,247 PM [test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-foo-config51 with journal batch size 1 02:56:06,247 PM [test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$B/member-1-shard-foo-config51/member-1-shard-foo-config51-notifier#-712136603 created and ready for shard:member-1-shard-foo-config51 02:56:06,247 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$B/member-1-shard-foo-config51/shard-member-1:snapshot-read#-1855006695] 02:56:06,256 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-foo-config51: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 02:56:06,256 PM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-foo-config51: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:56:06,256 PM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-foo-config51: onLeadershipChange, isLeader: false, hasLeader : false 02:56:06,256 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-foo-config51: onStateChanged: Closing all transaction chains because shard member-1-shard-foo-config51 is no longer the leader 02:56:06,256 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-foo-config51 (Follower): Received TimeoutNow - switching to Candidate 02:56:06,256 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-foo-config51 (Candidate): Election: Candidate has following voting peers: [] 02:56:06,256 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-foo-config51: Set currentTerm=1, votedFor=member-1-shard-foo-config51 02:56:06,256 PM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-foo-config51 (Candidate): Starting new election term 1 02:56:06,256 PM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-foo-config51 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:56:06,256 PM [test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-1-shard-foo-config51 (Follower): Closing 02:56:06,256 PM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-foo-config51: onLeadershipChange, isLeader: false, hasLeader : false 02:56:06,256 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-foo-config51: onStateChanged: Closing all transaction chains because shard member-1-shard-foo-config51 is no longer the leader 02:56:06,257 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config51: Initializing shard [foo] 02:56:06,257 PM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config51 , received role change from null to Follower 02:56:06,257 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardInformation - Shard member-1-shard-foo-config51 is initialized 02:56:06,257 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardInformation - Shard member-1-shard-foo-config51 is initialized - notifying 1 OnShardInitialized callbacks 02:56:06,257 PM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config51 , received role change from Follower to Candidate 02:56:06,257 PM [main] [INFO] ShardManagerTest - testOnCreateShardWithNoInitialSchemaContext ending 02:56:06,257 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-foo-config51 (Candidate): Received ElectionTimeout 02:56:06,257 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-foo-config51 (Leader): Election: Leader has following peers: [] 02:56:06,257 PM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-foo-config51 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 02:56:06,257 PM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-foo-config51: onLeadershipChange, isLeader: true, hasLeader : true 02:56:06,257 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-foo-config51: became leader with frontend state for [] 02:56:06,258 PM [test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config51 , received role change from Candidate to Leader 02:56:06,258 PM [test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config51 , registered listener akka://test/user/$B 02:56:06,258 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config51: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-foo-config51, leaderId=member-1-shard-foo-config51, leaderPayloadVersion=13] 02:56:06,258 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config51: Received role changed for member-1-shard-foo-config51 from Candidate to Leader 02:56:06,258 PM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config51: All Shards are ready - data store config51 is ready 02:56:06,359 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$B#1763080893] 02:56:06,370 PM [test-shard-dispatcher-18] [INFO] Shard - Stopping Shard member-1-shard-foo-config51 02:56:06,370 PM [test-shard-dispatcher-18] [DEBUG] MessageSlicer - member-1-shard-foo-config51 (Leader)_slicer-id-218: Closing 02:56:06,370 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-foo-config51: Aborting 0 pending queued transactions 02:56:06,371 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Stopping ShardManager shard-manager-config51 02:56:06,373 PM [main] [INFO] ShardManagerTest - testGetSnapshot starting 02:56:06,375 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Starting ShardManager shard-manager-config52 02:56:06,376 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config52 02:56:06,376 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config52: Creating local shard: member-1-shard-shard2-config52 02:56:06,377 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config52: Creating local shard: member-1-shard-shard1-config52 02:56:06,388 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config52: onGetSnapshot 02:56:06,388 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:56:06,388 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-shard2-config52 02:56:06,389 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-shard1-config52 02:56:06,389 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config52: findLocalShard : shard1 02:56:06,389 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config52: Scheduling 600 ms timer to wait for shard ShardInformation [shardId=member-1-shard-shard1-config52, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 02:56:06,393 PM [test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-shard2-config52, persistent : true 02:56:06,394 PM [test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-shard2-config52 with journal batch size 1 02:56:06,394 PM [test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-shard1-config52, persistent : true 02:56:06,395 PM [test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-shard1-config52 with journal batch size 1 02:56:06,395 PM [test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$m/member-1-shard-shard2-config52/member-1-shard-shard2-config52-notifier#-596685769 created and ready for shard:member-1-shard-shard2-config52 02:56:06,395 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$m/member-1-shard-shard2-config52/shard-member-1:snapshot-read#659027335] 02:56:06,395 PM [test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$m/member-1-shard-shard1-config52/member-1-shard-shard1-config52-notifier#1092535579 created and ready for shard:member-1-shard-shard1-config52 02:56:06,395 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$m/member-1-shard-shard1-config52/shard-member-1:snapshot-read#-965784841] 02:56:06,403 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard2-config52: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 02:56:06,403 PM [test-shard-dispatcher-20] [INFO] Shard - member-1-shard-shard2-config52: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:56:06,403 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard1-config52: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 02:56:06,403 PM [test-shard-dispatcher-19] [INFO] Shard - member-1-shard-shard1-config52: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:56:06,403 PM [test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard2-config52: onLeadershipChange, isLeader: false, hasLeader : false 02:56:06,403 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard2-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-shard2-config52 is no longer the leader 02:56:06,403 PM [test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard1-config52: onLeadershipChange, isLeader: false, hasLeader : false 02:56:06,403 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard1-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-shard1-config52 is no longer the leader 02:56:06,404 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard2-config52 (Follower): Received TimeoutNow - switching to Candidate 02:56:06,404 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard1-config52 (Follower): Received TimeoutNow - switching to Candidate 02:56:06,404 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard1-config52 (Candidate): Election: Candidate has following voting peers: [] 02:56:06,404 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard2-config52 (Candidate): Election: Candidate has following voting peers: [] 02:56:06,404 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard1-config52: Set currentTerm=1, votedFor=member-1-shard-shard1-config52 02:56:06,404 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard2-config52: Set currentTerm=1, votedFor=member-1-shard-shard2-config52 02:56:06,404 PM [test-shard-dispatcher-19] [INFO] Shard - member-1-shard-shard1-config52 (Candidate): Starting new election term 1 02:56:06,404 PM [test-shard-dispatcher-20] [INFO] Shard - member-1-shard-shard2-config52 (Candidate): Starting new election term 1 02:56:06,404 PM [test-shard-dispatcher-19] [INFO] Shard - member-1-shard-shard1-config52 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:56:06,404 PM [test-shard-dispatcher-20] [INFO] Shard - member-1-shard-shard2-config52 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:56:06,404 PM [test-shard-dispatcher-19] [DEBUG] MessageAssembler - member-1-shard-shard1-config52 (Follower): Closing 02:56:06,404 PM [test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-1-shard-shard2-config52 (Follower): Closing 02:56:06,404 PM [test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard1-config52: onLeadershipChange, isLeader: false, hasLeader : false 02:56:06,404 PM [test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard2-config52: onLeadershipChange, isLeader: false, hasLeader : false 02:56:06,404 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard1-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-shard1-config52 is no longer the leader 02:56:06,404 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard2-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-shard2-config52 is no longer the leader 02:56:06,404 PM [test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config52 , received role change from null to Follower 02:56:06,404 PM [test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config52 , received role change from Follower to Candidate 02:56:06,404 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config52: Initializing shard [shard2] 02:56:06,404 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-shard2-config52 is initialized 02:56:06,404 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config52: Initializing shard [shard1] 02:56:06,404 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-shard1-config52 is initialized 02:56:06,404 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-shard1-config52 is initialized - notifying 1 OnShardInitialized callbacks 02:56:06,404 PM [test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config52 , received role change from null to Follower 02:56:06,404 PM [test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config52 , received role change from Follower to Candidate 02:56:06,404 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config52: findLocalShard : shard2 02:56:06,404 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config52: onGetSnapshot 02:56:06,405 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Expecting 2 shard snapshot replies 02:56:06,405 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config52 (Candidate): Received ElectionTimeout 02:56:06,405 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config52 (Leader): Election: Leader has following peers: [] 02:56:06,405 PM [test-shard-dispatcher-19] [INFO] Shard - member-1-shard-shard2-config52 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 02:56:06,405 PM [test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard2-config52: onLeadershipChange, isLeader: true, hasLeader : true 02:56:06,405 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config52: became leader with frontend state for [] 02:56:06,405 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config52: onGetSnapshot 02:56:06,405 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config52: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 02:56:06,405 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config52: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 02:56:06,405 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard1-config52 (Candidate): Received ElectionTimeout 02:56:06,405 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard1-config52 (Leader): Election: Leader has following peers: [] 02:56:06,405 PM [test-shard-dispatcher-19] [INFO] Shard - member-1-shard-shard1-config52 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 02:56:06,405 PM [test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard1-config52: onLeadershipChange, isLeader: true, hasLeader : true 02:56:06,405 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard1-config52: became leader with frontend state for [] 02:56:06,406 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard1-config52: onGetSnapshot 02:56:06,406 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard1-config52: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 02:56:06,406 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard1-config52: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 02:56:06,406 PM [test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config52 , received role change from Candidate to Leader 02:56:06,406 PM [test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config52 , registered listener akka://test/user/$$m 02:56:06,406 PM [test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config52: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-shard2-config52, leaderId=member-1-shard-shard2-config52, leaderPayloadVersion=13] 02:56:06,406 PM [test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config52: Received role changed for member-1-shard-shard2-config52 from Candidate to Leader 02:56:06,406 PM [test-akka.actor.default-dispatcher-14] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Received GetSnapshotReply [id=member-1-shard-shard2-config52, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@3d67a50b, electionTerm=1, electionVotedFor=member-1-shard-shard2-config52, ServerConfigPayload=null]] 02:56:06,406 PM [test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config52 , received role change from Candidate to Leader 02:56:06,406 PM [test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config52 , registered listener akka://test/user/$$m 02:56:06,406 PM [test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config52: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-shard1-config52, leaderId=member-1-shard-shard1-config52, leaderPayloadVersion=13] 02:56:06,406 PM [test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config52: Received role changed for member-1-shard-shard1-config52 from Candidate to Leader 02:56:06,406 PM [test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config52: All Shards are ready - data store config52 is ready 02:56:06,407 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Received GetSnapshotReply [id=member-1-shard-shard1-config52, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@2eb2fd4, electionTerm=1, electionVotedFor=member-1-shard-shard1-config52, ServerConfigPayload=null]] 02:56:06,407 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: All shard snapshots received 02:56:06,418 PM [test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config52: onAddShardReplica: AddShardReplica[ShardName=astronauts] 02:56:06,419 PM [test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config52: Sending AddServer message to peer akka://test@127.0.0.1:2565/system/testProbe-609#-2087674589 for shard member-1-shard-astronauts-config52 02:56:06,421 PM [test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-astronauts-config52, persistent : true 02:56:06,422 PM [test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-astronauts-config52 with journal batch size 1 02:56:06,422 PM [test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$m/member-1-shard-astronauts-config52/member-1-shard-astronauts-config52-notifier#1425683535 created and ready for shard:member-1-shard-astronauts-config52 02:56:06,422 PM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$m/member-1-shard-astronauts-config52/shard-member-1:snapshot-read#-1904562606] 02:56:06,422 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config52: Received AddServerReply{status=OK, leaderHint=} for shard astronauts from leader akka://test@127.0.0.1:2565/system/testProbe-609#-2087674589 02:56:06,422 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config52: Leader shard successfully added the replica shard astronauts 02:56:06,423 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-astronauts-config52 02:56:06,423 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config52: persisting the shard list [shard2, astronauts, shard1] 02:56:06,423 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config52: findLocalShard : astronauts 02:56:06,423 PM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config52: Scheduling 600 ms timer to wait for shard ShardInformation [shardId=member-1-shard-astronauts-config52, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 02:56:06,425 PM [test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config52 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 02:56:06,426 PM [test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config52: Successfully deleted prior snapshots 02:56:06,429 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config52: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 02:56:06,429 PM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-astronauts-config52: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:56:06,429 PM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config52: onLeadershipChange, isLeader: false, hasLeader : false 02:56:06,429 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-astronauts-config52 is no longer the leader 02:56:06,429 PM [test-shard-dispatcher-18] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 02:56:06,429 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config52: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 02:56:06,429 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config52: Re-initializing to Follower with previous leaderId null 02:56:06,429 PM [test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-1-shard-astronauts-config52 (Follower): Closing 02:56:06,429 PM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config52: onLeadershipChange, isLeader: false, hasLeader : false 02:56:06,429 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-astronauts-config52 is no longer the leader 02:56:06,429 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config52 (Follower): Received TimeoutNow - switching to Candidate 02:56:06,429 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config52 (Candidate): Election: Candidate has following voting peers: [] 02:56:06,429 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config52: Set currentTerm=1, votedFor=member-1-shard-astronauts-config52 02:56:06,430 PM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-astronauts-config52 (Candidate): Starting new election term 1 02:56:06,430 PM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-astronauts-config52 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:56:06,430 PM [test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-1-shard-astronauts-config52 (Follower): Closing 02:56:06,430 PM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config52: onLeadershipChange, isLeader: false, hasLeader : false 02:56:06,430 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-astronauts-config52 is no longer the leader 02:56:06,430 PM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config52: Initializing shard [astronauts] 02:56:06,430 PM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-astronauts-config52 is initialized 02:56:06,430 PM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-astronauts-config52 is initialized - notifying 1 OnShardInitialized callbacks 02:56:06,430 PM [test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config52 , received role change from null to Follower 02:56:06,430 PM [test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config52 , received role change from Follower to Candidate 02:56:06,430 PM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config52: onGetSnapshot 02:56:06,431 PM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Expecting 3 shard snapshot replies 02:56:06,431 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-astronauts-config52 (Candidate): Received ElectionTimeout 02:56:06,431 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-astronauts-config52 (Leader): Election: Leader has following peers: [] 02:56:06,431 PM [test-shard-dispatcher-20] [INFO] Shard - member-1-shard-astronauts-config52 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 02:56:06,431 PM [test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config52: onLeadershipChange, isLeader: true, hasLeader : true 02:56:06,431 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-astronauts-config52: became leader with frontend state for [] 02:56:06,431 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-astronauts-config52: onGetSnapshot 02:56:06,431 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-astronauts-config52: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 02:56:06,431 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-astronauts-config52: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 02:56:06,431 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard2-config52: onGetSnapshot 02:56:06,431 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard2-config52: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 02:56:06,431 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard2-config52: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 02:56:06,431 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard1-config52: onGetSnapshot 02:56:06,431 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard1-config52: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 02:56:06,431 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard1-config52: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 02:56:06,431 PM [test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config52 , received role change from Candidate to Leader 02:56:06,431 PM [test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config52 , registered listener akka://test/user/$$m 02:56:06,431 PM [test-akka.actor.default-dispatcher-21] [INFO] ShardManager - shard-manager-config52: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-astronauts-config52, leaderId=member-1-shard-astronauts-config52, leaderPayloadVersion=13] 02:56:06,431 PM [test-akka.actor.default-dispatcher-21] [INFO] ShardManager - shard-manager-config52: Received role changed for member-1-shard-astronauts-config52 from Candidate to Leader 02:56:06,431 PM [test-akka.actor.default-dispatcher-21] [INFO] ShardManager - shard-manager-config52: All Shards are ready - data store config52 is ready 02:56:06,432 PM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Received GetSnapshotReply [id=member-1-shard-astronauts-config52, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@2f7aa7bc, electionTerm=1, electionVotedFor=member-1-shard-astronauts-config52, ServerConfigPayload=null]] 02:56:06,432 PM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Received GetSnapshotReply [id=member-1-shard-shard2-config52, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@4d67aad3, electionTerm=1, electionVotedFor=member-1-shard-shard2-config52, ServerConfigPayload=null]] 02:56:06,432 PM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Received GetSnapshotReply [id=member-1-shard-shard1-config52, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@1e9f0d6d, electionTerm=1, electionVotedFor=member-1-shard-shard1-config52, ServerConfigPayload=null]] 02:56:06,432 PM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: All shard snapshots received 02:56:06,432 PM [main] [INFO] ShardManagerTest - testGetSnapshot ending 02:56:06,535 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$m] 02:56:06,546 PM [test-shard-dispatcher-18] [INFO] Shard - Stopping Shard member-1-shard-astronauts-config52 02:56:06,546 PM [test-shard-dispatcher-18] [DEBUG] MessageSlicer - member-1-shard-astronauts-config52 (Leader)_slicer-id-224: Closing 02:56:06,546 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config52: Aborting 0 pending queued transactions 02:56:06,546 PM [test-shard-dispatcher-24] [INFO] Shard - Stopping Shard member-1-shard-shard1-config52 02:56:06,546 PM [test-shard-dispatcher-24] [DEBUG] MessageSlicer - member-1-shard-shard1-config52 (Leader)_slicer-id-222: Closing 02:56:06,546 PM [test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-shard1-config52: Aborting 0 pending queued transactions 02:56:06,546 PM [test-shard-dispatcher-19] [INFO] Shard - Stopping Shard member-1-shard-shard2-config52 02:56:06,546 PM [test-shard-dispatcher-19] [DEBUG] MessageSlicer - member-1-shard-shard2-config52 (Leader)_slicer-id-221: Closing 02:56:06,546 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config52: Aborting 0 pending queued transactions 02:56:06,547 PM [test-akka.actor.default-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-config52 02:56:06,548 PM [main] [INFO] ShardManagerTest - testWhenMultipleShardsPresentSyncStatusMustBeTrueForAllShards starting 02:56:06,552 PM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config53 02:56:06,557 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config53 02:56:06,557 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config53: Creating local shard: member-1-shard-default-config53 02:56:06,557 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config53: Creating local shard: member-1-shard-astronauts-config53 02:56:06,558 PM [main] [INFO] ShardManager - shard-manager-config53: Received role changed for member-1-shard-default-config53 from Follower to Leader 02:56:06,558 PM [main] [INFO] ShardManager - shard-manager-config53: Received role changed for member-1-shard-astronauts-config53 from Follower to Leader 02:56:06,558 PM [main] [INFO] ShardManager - shard-manager-config53: Received role changed for member-1-shard-astronauts-config53 from Leader to Follower 02:56:06,558 PM [main] [INFO] ShardManager - shard-manager-config53 Received follower initial sync status for member-1-shard-astronauts-config53 status sync done true 02:56:06,558 PM [main] [INFO] ShardManagerTest - testWhenMultipleShardsPresentSyncStatusMustBeTrueForAllShards ending 02:56:06,659 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$n] 02:56:06,670 PM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config53 02:56:06,672 PM [main] [INFO] ShardManagerTest - testAddShardReplicaWithFindPrimaryTimeout starting 02:56:06,674 PM [test-akka.actor.default-dispatcher-23] [INFO] ShardManager - Starting ShardManager shard-manager-config54 02:56:06,675 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config54 02:56:06,684 PM [test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:56:06,684 PM [test-akka.actor.default-dispatcher-23] [INFO] ShardManager - shard-manager-config54: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://non-existent@127.0.0.1:5 02:56:06,684 PM [test-akka.actor.default-dispatcher-23] [INFO] ShardManager - shard-manager-config54: All Shards are ready - data store config54 is ready 02:56:06,684 PM [test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config54: onAddShardReplica: AddShardReplica[ShardName=astronauts] 02:56:06,685 PM [test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config54: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=astronauts, waitUntilReady=true] 02:56:06,685 PM [test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config54: findPrimary for astronauts forwarding to remote ShardManager akka://non-existent@127.0.0.1:5/user/shardmanager-config54, visitedAddresses: [akka://test@127.0.0.1:2550/user/shardmanager-config54] 02:56:06,704 PM [test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://non-existent@127.0.0.1:5], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:5,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 02:56:06,704 PM [test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://non-existent@127.0.0.1:5], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:5,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 02:56:06,903 PM [test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config54: Received failure from FindPrimary for shard astronauts akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://test/user/shardmanager-config54#1326948966]] after [200 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.FindPrimary]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 02:56:06,903 PM [main] [INFO] ShardManagerTest - testAddShardReplicaWithFindPrimaryTimeout ending 02:56:07,4 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/shardmanager-config54#1326948966] 02:56:07,15 PM [test-akka.actor.default-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-config54 02:56:07,17 PM [main] [INFO] ShardManagerTest - testOnCreateShard starting 02:56:07,19 PM [test-akka.actor.default-dispatcher-23] [INFO] ShardManager - Starting ShardManager shard-manager-config55 02:56:07,20 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config55 02:56:07,20 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 02:56:07,30 PM [test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config55: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=foo-ns, moduleName=foo-module, shardName=foo, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-5}, MemberName{name=member-6}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@1e271ace] 02:56:07,31 PM [test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config55 doCreateShard: shardId: member-1-shard-foo-config55, memberNames: [MemberName{name=member-1}, MemberName{name=member-5}, MemberName{name=member-6}], peerAddresses: {member-6-shard-foo-config55=null, member-5-shard-foo-config55=null}, isActiveMember: true 02:56:07,31 PM [test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config55: findLocalShard : foo 02:56:07,31 PM [test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config55: Scheduling 300000 ms timer to wait for shard ShardInformation [shardId=member-1-shard-foo-config55, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 02:56:07,34 PM [test-shard-dispatcher-19] [INFO] Shard - Shard created : member-1-shard-foo-config55, persistent : false 02:56:07,35 PM [test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$C/member-1-shard-foo-config55/member-1-shard-foo-config55-notifier#-696384247 created and ready for shard:member-1-shard-foo-config55 02:56:07,35 PM [test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$C/member-1-shard-foo-config55/shard-member-1:snapshot-read#953879469] 02:56:07,37 PM [test-shard-dispatcher-19] [INFO] Shard - Starting recovery for member-1-shard-foo-config55 with journal batch size 1 02:56:07,42 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-foo-config55: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 02:56:07,42 PM [test-shard-dispatcher-19] [INFO] Shard - member-1-shard-foo-config55: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:56:07,42 PM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config55: Initializing shard [foo] 02:56:07,42 PM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-foo-config55 is initialized 02:56:07,42 PM [test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-foo-config55: onLeadershipChange, isLeader: false, hasLeader : false 02:56:07,42 PM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-foo-config55 is initialized - notifying 1 OnShardInitialized callbacks 02:56:07,42 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-foo-config55: onStateChanged: Closing all transaction chains because shard member-1-shard-foo-config55 is no longer the leader 02:56:07,42 PM [test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config55 , received role change from null to Follower 02:56:07,42 PM [test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config55 , registered listener akka://test/user/$C 02:56:07,42 PM [test-akka.actor.default-dispatcher-22] [INFO] ShardManager - shard-manager-config55: Received role changed for member-1-shard-foo-config55 from null to Follower 02:56:07,43 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config55: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=foo-ns, moduleName=foo-module, shardName=foo, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-5}, MemberName{name=member-6}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@1e271ace] 02:56:07,43 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config55: Shard foo already exists 02:56:07,43 PM [main] [INFO] ShardManagerTest - testOnCreateShard ending 02:56:07,144 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$C#418759086] 02:56:07,155 PM [test-shard-dispatcher-19] [INFO] Shard - Stopping Shard member-1-shard-foo-config55 02:56:07,155 PM [test-shard-dispatcher-19] [DEBUG] MessageAssembler - member-1-shard-foo-config55 (Follower): Closing 02:56:07,155 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-foo-config55: Aborting 0 pending queued transactions 02:56:07,155 PM [test-akka.actor.default-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-config55 02:56:07,156 PM [test-akka.actor.default-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-config1 02:56:07,157 PM [test-akka.actor.default-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-config43 02:56:07,157 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config35 02:56:07,157 PM [test-akka.actor.default-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-config11 02:56:07,157 PM [test-akka.actor.default-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-config21 02:56:07,157 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Stopping ShardManager shard-manager-config46 02:56:07,158 PM [test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:56:07,158 PM [test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:56:07,160 PM [test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 14:56:07.161] [test-akka.actor.internal-dispatcher-5] [Cluster(akka://test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config