01:48:45,256 PM [test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:48:45,272 PM [test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://test@127.0.0.1:2565] with UID [6615766891713531785] 01:48:45,273 PM [test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://test@127.0.0.1:2565] - Starting up, Akka version [2.6.21] ... 01:48:45,288 PM [test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://test@127.0.0.1:2565] - Registered cluster JMX MBean [akka:type=Cluster] 01:48:45,288 PM [test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://test@127.0.0.1:2565] - Started up successfully 01:48:45,288 PM [test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://test@127.0.0.1:2565] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:45,288 PM [test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://test@127.0.0.1:2565] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:45,316 PM [test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$a#223305617] to Actor[akka://test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:45,319 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Starting ShardManager shard-manager-config1 01:48:45,330 PM [test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config1 01:48:45,331 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config1: Creating local shard: member-1-shard-default-config1 01:48:45,333 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:48:45,333 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config1 01:48:45,334 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config1: Initializing shard [default] 01:48:45,334 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-default-config1 is initialized 01:48:45,334 PM [test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config1: Received role changed for member-1-shard-default-config1 from null to Follower 01:48:45,335 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config1: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=default, meberVotingStatusMap={member-2=true}] 01:48:45,336 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config1: findLocalShard : default 01:48:45,339 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-default-config1=true}, serversVisited=[]] to local shard akka://test/user/member-1-shard-default-config1 01:48:45,341 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config1: Received ServerChangeReply{status=NO_LEADER} from local shard akka://test/user/member-1-shard-default-config1 01:48:45,341 PM [test-akka.actor.default-dispatcher-12] [WARN] ShardManager - shard-manager-config1: ChangeServersVotingStatus failed for shard default with status NO_LEADER 01:48:45,492 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-default-config1#1339653063] 01:48:45,505 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForShardLeader starting 01:48:45,509 PM [test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config2 01:48:45,509 PM [test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$b#-2072605802] to Actor[akka://test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:45,511 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config2 01:48:45,511 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config2: Creating local shard: member-1-shard-default-config2 01:48:45,520 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:48:45,520 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config2 01:48:45,520 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 01:48:45,526 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config2: Scheduling 10000 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config2, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 01:48:45,670 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config2: Initializing shard [default] 01:48:45,670 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-default-config2 is initialized 01:48:45,670 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-default-config2 is initialized - notifying 1 OnShardInitialized callbacks 01:48:45,820 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config2: Received role changed for member-1-shard-default-config2 from Candidate to Leader 01:48:45,820 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config2 is initialized - notifying 1 OnShardInitialized callbacks 01:48:45,971 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config2, leaderId=member-1-shard-default-config2, leaderPayloadVersion=13] 01:48:45,971 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config2 is ready - notifying 1 OnShardInitialized callbacks 01:48:45,971 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config2: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://test@127.0.0.1:2565/user/member-1-shard-default-config#1064427565, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$qaTj6l0k@35b3c108] 01:48:45,972 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config2: All Shards are ready - data store config2 is ready 01:48:46,172 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForShardLeader ending 01:48:46,273 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$b#-2072605802] 01:48:46,284 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config2 01:48:46,289 PM [test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://test/user/$$g] to Actor[akka://test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:46,289 PM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config3 01:48:46,292 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config3 01:48:46,292 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config3: Creating local shard: member-1-shard-default-config3 01:48:46,293 PM [main] [INFO] ShardManager - shard-manager-config3: Received role changed for member-1-shard-default-config3 from Follower to Candidate 01:48:46,293 PM [main] [INFO] ShardManager - shard-manager-config3 Received follower initial sync status for member-1-shard-default-config3 status sync done true 01:48:46,395 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$g] 01:48:46,406 PM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config3 01:48:46,414 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:46,422 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-643887112788250267] 01:48:46,422 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:48:46,426 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:46,426 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:48:46,426 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:46,426 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:46,453 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config4 01:48:46,457 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:48:46,458 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config4] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:46,459 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:48:46,466 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:46,469 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:48:46,470 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config4 01:48:46,471 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config4: Creating local shard: member-1-shard-default-config4 01:48:46,471 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config4: Creating local shard: member-1-shard-astronauts-config4 01:48:46,485 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config4: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:46,485 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-default-config4 01:48:46,485 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-astronauts-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-astronauts-config4 01:48:46,502 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [4173470307224972384] 01:48:46,502 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:48:46,502 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:46,502 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:48:46,511 PM [main] [ERROR] ShardManagerTest - Mock Shard Leader Actor : TestActor[akka://cluster-test/user/member-2-shard-default-config4] 01:48:46,513 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:46,513 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:46,516 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config4 01:48:46,517 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config4] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:46,522 PM [main] [ERROR] ShardManagerTest - Forwarding actor : Actor[akka://cluster-test/user/shardmanager-config4/member-2-shard-default-config4#26372580] 01:48:46,533 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:48:46,533 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config4 01:48:46,533 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-default-config4 01:48:46,533 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-default-config4 to akka://cluster-test/user/member-1-shard-default-config 01:48:46,541 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config4: updated peer member-1-shard-default-config4 on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-default-config4 on shard member-1-shard-default-config4 whose actor address is Actor[akka://cluster-test/user/member-1-shard-default-config#-1444084124] 01:48:46,541 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-astronauts-config4 01:48:46,541 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-astronauts-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-astronauts-config4 01:48:46,541 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-astronauts-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-astronauts-config4 to akka://cluster-test/user/member-1-shard-default-config 01:48:46,541 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config4: updated peer member-1-shard-astronauts-config4 on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-astronauts-config4 on shard member-1-shard-astronauts-config4 whose actor address is Actor[akka://cluster-test/user/member-1-shard-default-config#-1444084124] 01:48:46,541 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config4: Initializing shard [default] 01:48:46,541 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-default-config4 is initialized 01:48:46,541 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config4: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config4, leaderId=member-2-shard-default-config4, leaderPayloadVersion=12] 01:48:46,541 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config4: Received role changed for member-1-shard-default-config4 from Candidate to Follower 01:48:46,548 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config4 01:48:46,548 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config4: Creating local shard: member-2-shard-default-config4 01:48:46,548 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:48:46,548 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-2-shard-default-config4 01:48:46,548 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config4: Initializing shard [default] 01:48:46,548 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Shard member-2-shard-default-config4 is initialized 01:48:46,548 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config4: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-config4, leaderId=member-2-shard-default-config4, leaderPayloadVersion=12] 01:48:46,548 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config4: Received role changed for member-2-shard-default-config4 from Candidate to Leader 01:48:46,549 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config4: All Shards are ready - data store config4 is ready 01:48:46,894 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:48:46,945 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:48:46,946 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config4: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:46,946 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-default-config4 01:48:46,946 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-default-config4 to akka://cluster-test/user/member-2-shard-default-config4 01:48:46,946 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config4: All Shards are ready - data store config4 is ready 01:48:47,447 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:48:47,450 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config4: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:47,450 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config4 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-default-config4 01:48:47,450 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config4 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-default-config4 to akka://cluster-test/user/member-1-shard-default-config 01:48:47,450 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-astronauts-config4 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-astronauts-config4 01:48:47,450 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-astronauts-config4 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-astronauts-config4 to akka://cluster-test/user/member-1-shard-default-config 01:48:47,451 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config4: onRemoveShardReplica: RemoveShardReplica [shardName=default, memberName=MemberName{name=member-1}] 01:48:47,454 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config4: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 01:48:47,455 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config4: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-default-config4, primaryVersion=12] 01:48:47,455 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config4: Sending RemoveServer message to peer akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-default-config4 for shard member-1-shard-default-config4 01:48:47,467 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config4: Received RemoveServerReply{status=OK, leaderHint=member-2-shard-default-config4} for shard default 01:48:47,467 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config4: Leader shard successfully removed the replica shard default 01:48:47,534 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config4: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:47,534 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config4 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-default-config4 01:48:47,534 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config4 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-default-config4 to akka://cluster-test/user/member-2-shard-default-config4 01:48:47,534 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config4: All Shards are ready - data store config4 is ready 01:48:47,610 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-config4 01:48:47,611 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:47,611 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:47,655 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:47,656 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:47,699 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-config4 01:48:47,705 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:47,705 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:48,720 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:48:48.723] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:48,742 PM [test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$c#-1663792358] to Actor[akka://test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:48,742 PM [test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config5 01:48:48,744 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config5 01:48:48,744 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config5: Creating local shard: member-1-shard-default-config5 01:48:48,753 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:48:48,753 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config5 01:48:48,753 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config5: Initializing shard [default] 01:48:48,753 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config5 is initialized 01:48:48,905 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$c#-1663792358] 01:48:48,915 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config5 01:48:48,919 PM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config6 01:48:48,920 PM [test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://test/user/$$h] to Actor[akka://test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:48,924 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config6 01:48:48,924 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config6: Creating local shard: member-1-shard-default-config6 01:48:49,20 PM [main] [INFO] ShardManager - shard-manager-config6: Received role changed for member-1-shard-default-config6 from null to Follower 01:48:49,31 PM [main] [INFO] ShardManager - shard-manager-config6: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://test 01:48:49,31 PM [main] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config6 with address akka://test/user/shardmanager-config6/member-2-shard-default-config6 01:48:49,31 PM [main] [INFO] ShardManager - shard-manager-config6: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config6, leaderId=member-2-shard-default-config6, leaderPayloadVersion=13] 01:48:49,31 PM [main] [INFO] ShardManager - shard-manager-config6: All Shards are ready - data store config6 is ready 01:48:49,132 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$h] 01:48:49,143 PM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config6 01:48:49,148 PM [test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$d#-76341731] to Actor[akka://test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:49,148 PM [test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config7 01:48:49,152 PM [test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config7 01:48:49,158 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config7: onRemoveShardReplica: RemoveShardReplica [shardName=model-inventory, memberName=MemberName{name=member-1}] 01:48:49,159 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config7: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=model-inventory, waitUntilReady=true] 01:48:49,159 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config7: No shard found for model-inventory 01:48:49,159 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config7: Failed to find leader for shard model-inventory: received response: {} org.opendaylight.controller.cluster.datastore.exceptions.PrimaryNotFoundException: No primary shard found for model-inventory. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.findPrimary(ShardManager.java:1056) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:207) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManagerTest$TestShardManager.handleCommand(ShardManagerTest.java:2198) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:48:49,262 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$d#-76341731] 01:48:49,272 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Stopping ShardManager shard-manager-config7 01:48:49,274 PM [main] [INFO] ShardManagerTest - testAddShardReplicaWithPreExistingLocalReplicaLeader starting 01:48:49,276 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Starting ShardManager shard-manager-config8 01:48:49,276 PM [test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$e#1332984462] to Actor[akka://test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:49,278 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config8 01:48:49,278 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config8: Creating local shard: member-1-shard-default-config8 01:48:49,287 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:48:49,287 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config8 01:48:49,287 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config8: Initializing shard [default] 01:48:49,287 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardInformation - Shard member-1-shard-default-config8 is initialized 01:48:49,287 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - shard-manager-config8: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config8, leaderId=member-1-shard-default-config8, leaderPayloadVersion=13] 01:48:49,287 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - shard-manager-config8: Received role changed for member-1-shard-default-config8 from Candidate to Leader 01:48:49,287 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - shard-manager-config8: All Shards are ready - data store config8 is ready 01:48:49,288 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config8: onAddShardReplica: AddShardReplica[ShardName=default] 01:48:49,289 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config8: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 01:48:49,289 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config8: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://test@127.0.0.1:2565/user/member-1-shard-default-config#1088187332, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$qaTj6l0k@753d7dea] 01:48:49,290 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config8: Local shard default already exists 01:48:49,290 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config8: findLocalShard : default 01:48:49,290 PM [main] [INFO] ShardManagerTest - testAddShardReplicaWithPreExistingLocalReplicaLeader ending 01:48:49,392 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$e#1332984462] 01:48:49,402 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config8 01:48:49,404 PM [main] [INFO] ShardManagerTest - testOnReceiveFindLocalShardWaitForShardInitialized starting 01:48:49,407 PM [test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$f#-1472447348] to Actor[akka://test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:49,408 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Starting ShardManager shard-manager-config9 01:48:49,413 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config9 01:48:49,413 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config9: Creating local shard: member-1-shard-default-config9 01:48:49,418 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:48:49,418 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config9 01:48:49,418 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config9: findLocalShard : default 01:48:49,418 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config9: Scheduling 600 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config9, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 01:48:49,418 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config9: Initializing shard [default] 01:48:49,418 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-default-config9 is initialized 01:48:49,418 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-default-config9 is initialized - notifying 1 OnShardInitialized callbacks 01:48:49,421 PM [main] [INFO] ShardManagerTest - testOnReceiveFindLocalShardWaitForShardInitialized starting 01:48:49,523 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$f#-1472447348] 01:48:49,534 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config9 01:48:49,535 PM [main] [INFO] ShardManagerTest - testShardAvailabilityChangeOnMemberUnreachableAndLeadershipChange starting 01:48:49,543 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:49,552 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [8088095116905550600] 01:48:49,552 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:48:49,563 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:48:49,563 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:48:49,563 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:49,563 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:49,568 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config10 01:48:49,569 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:48:49,576 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:48:49,576 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config10] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:49,577 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:48:49,584 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-config10 01:48:49,584 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config10: Creating local shard: member-1-shard-default-config10 01:48:49,585 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] ShardManager - shard-manager-config10: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:49,585 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config10 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config10/member-1-shard-default-config10 01:48:49,586 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:48:49,598 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-8584948368721972933] 01:48:49,598 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:48:49,600 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:49,600 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:48:49,611 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:49,611 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:49,616 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Starting ShardManager shard-manager-config10 01:48:49,616 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config10] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:49,623 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:48:49,623 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config10 01:48:49,623 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config10 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config10/member-1-shard-default-config10 01:48:49,623 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config10 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config10/member-1-shard-default-config10 to akka://cluster-test/user/member-1-shard-default-config 01:48:49,623 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config10: updated peer member-1-shard-default-config10 on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config10/member-1-shard-default-config10 on shard member-1-shard-default-config10 whose actor address is Actor[akka://cluster-test/user/member-1-shard-default-config#1933384919] 01:48:49,623 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config10: Initializing shard [default] 01:48:49,623 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-default-config10 is initialized 01:48:49,623 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] ShardManager - shard-manager-config10: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config10, leaderId=member-2-shard-default-config10, leaderPayloadVersion=13] 01:48:49,623 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] ShardManager - shard-manager-config10: Received role changed for member-1-shard-default-config10 from Candidate to Follower 01:48:49,653 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - Recovery complete : shard-manager-config10 01:48:49,653 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config10: Creating local shard: member-2-shard-default-config10 01:48:49,654 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:48:49,654 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-2-shard-default-config10 01:48:49,654 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config10: Initializing shard [default] 01:48:49,654 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardInformation - Shard member-2-shard-default-config10 is initialized 01:48:49,654 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config10: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-config10, leaderId=member-2-shard-default-config10, leaderPayloadVersion=13] 01:48:49,654 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config10: Received role changed for member-2-shard-default-config10 from Candidate to Leader 01:48:49,654 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config10: All Shards are ready - data store config10 is ready 01:48:49,695 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:48:49,702 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:48:49,702 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardManager - shard-manager-config10: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:49,702 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config10 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config10/member-1-shard-default-config10 01:48:49,702 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config10 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config10/member-1-shard-default-config10 to akka://cluster-test/user/member-2-shard-default-config 01:48:49,702 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardManager - shard-manager-config10: All Shards are ready - data store config10 is ready 01:48:50,581 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:48:50,582 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] ShardManager - shard-manager-config10: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:50,582 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config10 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config10/member-2-shard-default-config10 01:48:50,582 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config10 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config10/member-2-shard-default-config10 to akka://cluster-test/user/member-1-shard-default-config 01:48:50,582 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] ShardManager - shard-manager-config10: All Shards are ready - data store config10 is ready 01:48:50,582 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config10: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 01:48:50,582 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config10: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config10/member-2-shard-default-config10, primaryVersion=13] 01:48:50,583 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 01:48:50,583 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - Marking Leader member-2-shard-default-config10 as unavailable. 01:48:50,583 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config10: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 01:48:50,583 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config10: Scheduling 1200 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config10, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=member-2-shard-default-config10, activeMember=true] 01:48:50,627 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config10: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:50,627 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config10 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config10/member-2-shard-default-config10 01:48:50,627 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config10 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config10/member-2-shard-default-config10 to akka://cluster-test/user/member-2-shard-default-config 01:48:50,627 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config10: All Shards are ready - data store config10 is ready 01:48:51,799 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config10: Received ShardNotInitializedTimeout message for shard default 01:48:51,799 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config10: Returning NoShardLeaderException for shard default 01:48:51,800 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] ShardManager - shard-manager-config10: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config10, leaderId=member-1-shard-default-config10, leaderPayloadVersion=13] 01:48:51,800 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] ShardManager - shard-manager-config10: All Shards are ready - data store config10 is ready 01:48:51,800 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] ShardManager - shard-manager-config10: Received role changed for member-1-shard-default-config10 from Follower to Leader 01:48:51,800 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] ShardManager - shard-manager-config10: All Shards are ready - data store config10 is ready 01:48:51,800 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config10: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 01:48:51,800 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config10: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/member-1-shard-default-config#1933384919, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$qaTj6l0k@45e42125] 01:48:51,800 PM [main] [INFO] ShardManagerTest - testShardAvailabilityChangeOnMemberUnreachableAndLeadershipChange ending 01:48:51,902 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config10 01:48:51,905 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:51,905 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:51,914 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:51,914 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:51,916 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:48:51,933 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - Stopping ShardManager shard-manager-config10 01:48:51,934 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:51,934 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/11/2024 13:48:52.951] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:52,961 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config11 01:48:52,961 PM [test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://test/user/shardmanager-config11] to Actor[akka://test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:52,963 PM [test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config11 01:48:52,982 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:48:52,982 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config11: onAddShardReplica: AddShardReplica[ShardName=astronauts] 01:48:52,985 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config11: Sending AddServer message to peer akka://test@127.0.0.1:2565/system/testProbe-552#1179856179 for shard member-1-shard-astronauts-config11 01:48:52,996 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config11: onRemoveShardReplica: RemoveShardReplica [shardName=astronauts, memberName=MemberName{name=member-2}] 01:48:52,997 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config11: A shard replica operation for astronauts is already in progress 01:48:53,101 PM [main] [INFO] ShardManagerTest - testPerShardDatastoreContext starting 01:48:53,109 PM [test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$g#-45822420] to Actor[akka://test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:53,109 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Starting ShardManager shard-manager-config12 01:48:53,110 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config12 01:48:53,110 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config12: Creating local shard: member-1-shard-default-config12 01:48:53,111 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config12: Creating local shard: member-1-shard-topology-config12 01:48:53,133 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:48:53,133 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config12 01:48:53,133 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-topology-config12 01:48:53,134 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-default-config12 01:48:53,134 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-topology-config12 01:48:53,184 PM [main] [INFO] ShardManagerTest - testPerShardDatastoreContext ending 01:48:53,286 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/default1#2080401146] 01:48:53,297 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/topology2#-2024298859] 01:48:53,297 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$g#-45822420] 01:48:53,297 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Stopping ShardManager shard-manager-config12 01:48:53,299 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForLocalLeaderShard starting 01:48:53,305 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config13 01:48:53,306 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config13 01:48:53,306 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config13: Creating local shard: member-1-shard-default-config13 01:48:53,315 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:48:53,315 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config13 01:48:53,316 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config13: Initializing shard [default] 01:48:53,316 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardInformation - Shard member-1-shard-default-config13 is initialized 01:48:53,316 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - shard-manager-config13: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config13, leaderId=member-1-shard-default-config13, leaderPayloadVersion=13] 01:48:53,366 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - shard-manager-config13: Received role changed for member-1-shard-default-config13 from Candidate to Leader 01:48:53,366 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - shard-manager-config13: All Shards are ready - data store config13 is ready 01:48:53,367 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config13: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 01:48:53,367 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config13: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://test@127.0.0.1:2565/user/member-1-shard-default-config#-593319789, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$qaTj6l0k@45fda0cf] 01:48:53,367 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForLocalLeaderShard ending 01:48:53,468 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$h#-1668359069] 01:48:53,479 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Stopping ShardManager shard-manager-config13 01:48:53,481 PM [main] [INFO] ShardManagerTest - testShardAvailabilityChangeOnMemberWithNameContainedInLeaderIdUnreachable starting 01:48:53,488 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:53,498 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2562] with UID [-5324307945767881447] 01:48:53,498 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Starting up, Akka version [2.6.21] ... 01:48:53,500 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Registered cluster JMX MBean [akka:type=Cluster] 01:48:53,500 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Started up successfully 01:48:53,512 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:53,512 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:53,515 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Node [akka://cluster-test@127.0.0.1:2562] is JOINING itself (with roles [member-256, dc-default], version [0.0.0]) and forming new cluster 01:48:53,515 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - is the new leader among reachable nodes (more leaders may exist) 01:48:53,515 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config14] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:53,515 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Leader is moving node [akka://cluster-test@127.0.0.1:2562] to [Up] 01:48:53,517 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config14 01:48:53,522 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config14 01:48:53,522 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config14: Creating local shard: member-256-shard-default-config14 01:48:53,523 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config14: Received MemberUp: memberName: MemberName{name=member-256}, address: akka://cluster-test@127.0.0.1:2562 01:48:53,523 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-256-shard-default-config14 with address akka://cluster-test@127.0.0.1:2562/user/shardmanager-config14/member-256-shard-default-config14 01:48:53,526 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:53,533 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [1048291399860114411] 01:48:53,533 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:48:53,539 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:53,539 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:48:53,544 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:53,544 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:53,554 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config14] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:53,558 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config14 01:48:53,559 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:48:53,559 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Creating Shard member-256-shard-default-config14 01:48:53,559 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardInformation - updatePeerAddress for peer member-256-shard-default-config14 with address akka://cluster-test@127.0.0.1:2562/user/shardmanager-config14/member-256-shard-default-config14 01:48:53,559 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-256-shard-default-config14 with address akka://cluster-test@127.0.0.1:2562/user/shardmanager-config14/member-256-shard-default-config14 to akka://cluster-test/user/member-256-shard-default-config 01:48:53,559 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config14: updated peer member-256-shard-default-config14 on member MemberName{name=member-256} with address akka://cluster-test@127.0.0.1:2562/user/shardmanager-config14/member-256-shard-default-config14 on shard member-256-shard-default-config14 whose actor address is Actor[akka://cluster-test/user/member-256-shard-default-config#925287835] 01:48:53,559 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config14: Initializing shard [default] 01:48:53,559 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Shard member-256-shard-default-config14 is initialized 01:48:53,559 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config14: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-256-shard-default-config14, leaderId=member-256-shard-default-config14, leaderPayloadVersion=13] 01:48:53,559 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config14: Received role changed for member-256-shard-default-config14 from Candidate to Leader 01:48:53,559 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config14: All Shards are ready - data store config14 is ready 01:48:53,569 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - Recovery complete : shard-manager-config14 01:48:53,569 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config14: Creating local shard: member-2-shard-default-config14 01:48:53,570 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:48:53,570 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-2-shard-default-config14 01:48:53,570 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config14: Initializing shard [default] 01:48:53,570 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardInformation - Shard member-2-shard-default-config14 is initialized 01:48:53,570 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config14: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-config14, leaderId=member-256-shard-default-config14, leaderPayloadVersion=13] 01:48:53,570 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config14: Received role changed for member-2-shard-default-config14 from Candidate to Follower 01:48:53,601 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:48:53,604 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2562] 01:48:53,604 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config14: Received MemberUp: memberName: MemberName{name=member-256}, address: akka://cluster-test@127.0.0.1:2562 01:48:53,605 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-256-shard-default-config14 with address akka://cluster-test@127.0.0.1:2562/user/shardmanager-config14/member-256-shard-default-config14 01:48:53,605 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-256-shard-default-config14 with address akka://cluster-test@127.0.0.1:2562/user/shardmanager-config14/member-256-shard-default-config14 to akka://cluster-test/user/member-2-shard-default-config 01:48:53,605 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config14: All Shards are ready - data store config14 is ready 01:48:54,517 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:48:54,518 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config14: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:54,518 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config14 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config14/member-2-shard-default-config14 01:48:54,518 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config14 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config14/member-2-shard-default-config14 to akka://cluster-test/user/member-256-shard-default-config 01:48:54,518 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config14: All Shards are ready - data store config14 is ready 01:48:54,519 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config14: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 01:48:54,519 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config14: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2562/user/member-256-shard-default-config#925287835, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$qaTj6l0k@4c3ecdeb] 01:48:54,524 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 01:48:54,524 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config14: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 01:48:54,525 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config14: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2562/user/member-256-shard-default-config#925287835, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$qaTj6l0k@4c3ecdeb] 01:48:54,526 PM [main] [INFO] ShardManagerTest - testShardAvailabilityChangeOnMemberWithNameContainedInLeaderIdUnreachable ending 01:48:54,568 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config14: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:54,568 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config14 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config14/member-2-shard-default-config14 01:48:54,568 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config14 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config14/member-2-shard-default-config14 to akka://cluster-test/user/member-2-shard-default-config 01:48:54,568 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config14: All Shards are ready - data store config14 is ready 01:48:54,636 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config14 01:48:54,637 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:54,637 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:54,645 PM [cluster-test-akka.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:54,645 PM [cluster-test-akka.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:54,667 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - Stopping ShardManager shard-manager-config14 01:48:54,668 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:54,668 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:55,576 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - is the new leader among reachable nodes (more leaders may exist) 01:48:55,694 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:48:55.696] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:55,714 PM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config15 01:48:55,716 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config15 01:48:55,716 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config15: Creating local shard: member-1-shard-default-config15 01:48:55,717 PM [main] [INFO] ShardManager - shard-manager-config15: Received role changed for member-1-shard-default-config15 from Follower to Leader 01:48:55,818 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$i] 01:48:55,829 PM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config15 01:48:55,834 PM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config16 01:48:55,835 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config16 01:48:55,835 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config16: Creating local shard: member-1-shard-default-config16 01:48:55,935 PM [main] [INFO] ShardManager - shard-manager-config16: Received role changed for member-1-shard-default-config16 from Candidate to Leader 01:48:55,935 PM [main] [INFO] ShardManager - shard-manager-config16: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config16, leaderId=member-1-shard-default-config16, leaderPayloadVersion=13] 01:48:55,935 PM [main] [INFO] ShardManager - shard-manager-config16: All Shards are ready - data store config16 is ready 01:48:56,37 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$j] 01:48:56,48 PM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config16 01:48:56,50 PM [main] [INFO] ShardManagerTest - testOnCreateShardWithLocalMemberNotInShardConfig starting 01:48:56,52 PM [test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config17 01:48:56,54 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config17 01:48:56,54 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:48:56,67 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config17: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=foo-ns, moduleName=foo-module, shardName=foo, shardMemberNames=[MemberName{name=member-5}, MemberName{name=member-6}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@53327443] 01:48:56,69 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config17 doCreateShard: shardId: member-1-shard-foo-config17, memberNames: [MemberName{name=member-5}, MemberName{name=member-6}], peerAddresses: {}, isActiveMember: false 01:48:56,71 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config17: findLocalShard : foo 01:48:56,71 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config17: Scheduling 60000 ms timer to wait for shard ShardInformation [shardId=member-1-shard-foo-config17, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=false] 01:48:56,72 PM [test-shard-dispatcher-15] [INFO] Shard - Shard created : member-1-shard-foo-config17, persistent : true 01:48:56,74 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$i/member-1-shard-foo-config17/shard-member-1:snapshot-read#671457132] 01:48:56,74 PM [test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$i/member-1-shard-foo-config17/member-1-shard-foo-config17-notifier#389694071 created and ready for shard:member-1-shard-foo-config17 01:48:56,74 PM [test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-1-shard-foo-config17 with journal batch size 1 01:48:56,77 PM [test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-foo-config17: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 01:48:56,77 PM [test-shard-dispatcher-15] [INFO] Shard - member-1-shard-foo-config17: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:56,77 PM [test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-foo-config17: onLeadershipChange, isLeader: false, hasLeader : false 01:48:56,77 PM [test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-foo-config17: onStateChanged: Closing all transaction chains because shard member-1-shard-foo-config17 is no longer the leader 01:48:56,77 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config17: Initializing shard [foo] 01:48:56,77 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-foo-config17 is initialized 01:48:56,77 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-foo-config17 is initialized - notifying 1 OnShardInitialized callbacks 01:48:56,77 PM [test-shard-dispatcher-15] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:48:56,77 PM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config17 , received role change from null to Follower 01:48:56,77 PM [main] [INFO] ShardManagerTest - testOnCreateShardWithLocalMemberNotInShardConfig ending 01:48:56,78 PM [test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config17 , registered listener akka://test/user/$i 01:48:56,78 PM [test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config17: Received role changed for member-1-shard-foo-config17 from null to Follower 01:48:56,179 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$i#-2043889749] 01:48:56,190 PM [test-shard-dispatcher-15] [INFO] Shard - Stopping Shard member-1-shard-foo-config17 01:48:56,190 PM [test-shard-dispatcher-15] [DEBUG] MessageAssembler - member-1-shard-foo-config17 (Follower): Closing 01:48:56,190 PM [test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-foo-config17: Aborting 0 pending queued transactions 01:48:56,191 PM [test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config17 01:48:56,192 PM [main] [INFO] ShardManagerTest - testServerRemovedShardActorNotRunning starting 01:48:56,194 PM [test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config18 01:48:56,196 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config18 01:48:56,196 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config18: Creating local shard: member-1-shard-default-config18 01:48:56,196 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config18: Creating local shard: member-1-shard-people-config18 01:48:56,207 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config18: findLocalShard : people 01:48:56,207 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config18: Returning NotInitializedException for shard people 01:48:56,207 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config18: findLocalShard : default 01:48:56,207 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config18: Returning NotInitializedException for shard default 01:48:56,209 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config18 : Local Shard replica for shard default has been removed 01:48:56,209 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config18: persisting the shard list [people] 01:48:56,209 PM [main] [INFO] ShardManagerTest - testServerRemovedShardActorNotRunning ending 01:48:56,218 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config18 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 01:48:56,224 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config18: Successfully deleted prior snapshots 01:48:56,310 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$k] 01:48:56,321 PM [test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config18 01:48:56,325 PM [test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config19 01:48:56,326 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config19 01:48:56,326 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config19: Creating local shard: member-1-shard-default-config19 01:48:56,335 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:48:56,335 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config19 01:48:56,336 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config19: Initializing shard [default] 01:48:56,336 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-default-config19 is initialized 01:48:56,336 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config19: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 01:48:56,336 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config19: Returning NoShardLeaderException for shard default 01:48:56,437 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$j#-1184324218] 01:48:56,447 PM [test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config19 01:48:56,449 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForFollowerShardWithNoInitialLeaderId starting 01:48:56,451 PM [test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config20 01:48:56,452 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config20 01:48:56,452 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config20: Creating local shard: member-1-shard-default-config20 01:48:56,461 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:48:56,461 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config20 01:48:56,461 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config20: Initializing shard [default] 01:48:56,461 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-default-config20 is initialized 01:48:56,461 PM [test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config20: Received role changed for member-1-shard-default-config20 from Candidate to Follower 01:48:56,461 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config20: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 01:48:56,461 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config20: Returning NoShardLeaderException for shard default 01:48:56,462 PM [test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config20: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config20, leaderId=member-1-shard-default-config20, leaderPayloadVersion=13] 01:48:56,462 PM [test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config20: All Shards are ready - data store config20 is ready 01:48:56,462 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config20: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 01:48:56,462 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config20: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://test@127.0.0.1:2565/user/member-1-shard-default-config#-1591470725, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$qaTj6l0k@1cc6e135] 01:48:56,462 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForFollowerShardWithNoInitialLeaderId starting 01:48:56,563 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$k#-770790051] 01:48:56,574 PM [test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config20 01:48:56,578 PM [test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config21 01:48:56,579 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config21 01:48:56,599 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:48:56,599 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config21: onAddShardReplica: AddShardReplica[ShardName=astronauts] 01:48:56,600 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config21: Sending AddServer message to peer akka://test@127.0.0.1:2565/system/testProbe-570#-929124297 for shard member-1-shard-astronauts-config21 01:48:56,610 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config21: onAddShardReplica: AddShardReplica[ShardName=astronauts] 01:48:56,611 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config21: A shard replica operation for astronauts is already in progress 01:48:56,713 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithCandidateShard starting 01:48:56,716 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config22 01:48:56,717 PM [test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config22 01:48:56,717 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config22: Creating local shard: member-1-shard-default-config22 01:48:56,726 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:48:56,726 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config22 01:48:56,726 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config22: Initializing shard [default] 01:48:56,726 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config22 is initialized 01:48:56,727 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config22: Received role changed for member-1-shard-default-config22 from null to Candidate 01:48:56,727 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config22: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 01:48:56,727 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config22: Scheduling 1200 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config22, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Candidate, leaderId=null, activeMember=true] 01:48:57,944 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config22: Received ShardNotInitializedTimeout message for shard default 01:48:57,944 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config22: Returning NoShardLeaderException for shard default 01:48:57,944 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithCandidateShard ending 01:48:58,46 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$l#-1627465667] 01:48:58,56 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config22 01:48:58,58 PM [main] [INFO] ShardManagerTest - testAddShardReplicaWithAddServerReplyFailure starting 01:48:58,61 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config23 01:48:58,63 PM [test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config23 01:48:58,82 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:48:58,92 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config23: onAddShardReplica: AddShardReplica[ShardName=astronauts] 01:48:58,93 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config23: Sending AddServer message to peer akka://test@127.0.0.1:2565/system/testProbe-575#1200748243 for shard member-1-shard-astronauts-config23 01:48:58,94 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config23: Received AddServerReply{status=TIMEOUT} for shard astronauts from leader akka://test@127.0.0.1:2565/system/testProbe-575#1200748243 01:48:58,94 PM [test-akka.actor.default-dispatcher-12] [WARN] ShardManager - shard-manager-config23: Leader failed to add shard replica astronauts with status TIMEOUT 01:48:58,94 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config23: findLocalShard : astronauts 01:48:58,94 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config23: Local shard astronauts not found - shards present: [] 01:48:58,94 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config23: onAddShardReplica: AddShardReplica[ShardName=astronauts] 01:48:58,95 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config23: Sending AddServer message to peer akka://test@127.0.0.1:2565/system/testProbe-575#1200748243 for shard member-1-shard-astronauts-config23 01:48:58,95 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config23: Received AddServerReply{status=NO_LEADER} for shard astronauts from leader akka://test@127.0.0.1:2565/system/testProbe-575#1200748243 01:48:58,95 PM [test-akka.actor.default-dispatcher-12] [WARN] ShardManager - shard-manager-config23: Leader failed to add shard replica astronauts with status NO_LEADER 01:48:58,96 PM [main] [INFO] ShardManagerTest - testAddShardReplicaWithAddServerReplyFailure ending 01:48:58,197 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-astronauts-config#-1990256785] 01:48:58,208 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/shardmanager-config23] 01:48:58,208 PM [test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config23 01:48:58,212 PM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config24 01:48:58,213 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config24 01:48:58,214 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config24: Creating local shard: member-1-shard-default-config24 01:48:58,313 PM [main] [INFO] ShardManager - shard-manager-config24: Received role changed for member-1-shard-default-config24 from Candidate to Follower 01:48:58,313 PM [main] [INFO] ShardManager - shard-manager-config24 Received follower initial sync status for member-1-shard-default-config24 status sync done true 01:48:58,313 PM [main] [INFO] ShardManager - shard-manager-config24 Received follower initial sync status for member-1-shard-default-config24 status sync done false 01:48:58,416 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$l] 01:48:58,427 PM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config24 01:48:58,431 PM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config25 01:48:58,432 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config25 01:48:58,432 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config25: Creating local shard: member-1-shard-default-config25 01:48:58,433 PM [main] [INFO] ShardManager - shard-manager-config25: Received role changed for unknown from Candidate to Leader 01:48:58,535 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$m] 01:48:58,545 PM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config25 01:48:58,547 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithUninitializedShard starting 01:48:58,550 PM [test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config26 01:48:58,554 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config26 01:48:58,554 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config26: Creating local shard: member-1-shard-default-config26 01:48:58,560 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:48:58,560 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config26 01:48:58,560 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config26: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 01:48:58,560 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config26: Scheduling 600 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config26, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 01:48:59,174 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config26: Received ShardNotInitializedTimeout message for shard default 01:48:59,174 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config26: Returning NotInitializedException for shard default 01:48:59,175 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config26: Initializing shard [default] 01:48:59,175 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-default-config26 is initialized 01:48:59,374 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithUninitializedShard ending 01:48:59,476 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$m#-1290889530] 01:48:59,487 PM [test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config26 01:48:59,494 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Starting ShardManager shard-manager-config27 01:48:59,495 PM [test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config27 01:48:59,495 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config27: Creating local shard: member-1-shard-default-config27 01:48:59,505 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:48:59,505 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config27 01:48:59,505 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config27: Initializing shard [default] 01:48:59,505 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardInformation - Shard member-1-shard-default-config27 is initialized 01:48:59,505 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config27: findLocalShard : default 01:48:59,607 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$n#1199582129] 01:48:59,617 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Stopping ShardManager shard-manager-config27 01:48:59,619 PM [main] [INFO] ShardManagerTest - testRegisterForShardLeaderChanges starting 01:48:59,621 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Starting ShardManager shard-manager-config28 01:48:59,622 PM [test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config28 01:48:59,623 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config28: Creating local shard: member-1-shard-default-config28 01:48:59,632 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:48:59,632 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config28 01:48:59,632 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config28: Initializing shard [default] 01:48:59,632 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config28 is initialized 01:48:59,632 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config28: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7087785 01:48:59,633 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config28: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config28, leaderId=member-1-shard-default-config28, leaderPayloadVersion=13] 01:48:59,646 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config28: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config28, leaderId=member-1-shard-default-config28, leaderPayloadVersion=13] 01:49:00,147 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config28: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config28, leaderId=null, leaderPayloadVersion=13] 01:49:00,158 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config28: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config28, leaderId=member-2-shard-default-config28, leaderPayloadVersion=13] 01:49:00,168 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config28: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config28, leaderId=member-1-shard-default-config28, leaderPayloadVersion=13] 01:49:00,669 PM [main] [INFO] ShardManagerTest - testRegisterForShardLeaderChanges ending 01:49:00,770 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$o#182690480] 01:49:00,781 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config28 01:49:00,785 PM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config29 01:49:00,786 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config29 01:49:00,786 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config29: Creating local shard: member-1-shard-default-config29 01:49:00,888 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$n] 01:49:00,898 PM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config29 01:49:00,902 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config30 01:49:00,903 PM [test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config30 01:49:00,903 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config30: Creating local shard: member-1-shard-default-config30 01:49:00,913 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:49:00,913 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config30 01:49:00,913 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config30: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=non-existent, waitUntilReady=false] 01:49:00,913 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config30: No shard found for non-existent 01:49:01,14 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$p#247800874] 01:49:01,25 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config30 01:49:01,32 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config31 01:49:01,36 PM [test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config31 01:49:01,43 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config31: onAddShardReplica: AddShardReplica[ShardName=model-inventory] 01:49:01,43 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config31: No module configuration exists for shard model-inventory 01:49:01,144 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$q#388035029] 01:49:01,155 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config31 01:49:01,156 PM [main] [INFO] ShardManagerTest - testAddShardReplicaWithPreExistingReplicaInRemoteShardLeader starting 01:49:01,158 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config32 01:49:01,159 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config32 01:49:01,159 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config32: Creating local shard: member-1-shard-default-config32 01:49:01,268 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:49:01,268 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config32 01:49:01,268 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config32: Initializing shard [default] 01:49:01,268 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config32 is initialized 01:49:01,268 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config32: Received MemberUp: memberName: MemberName{name=leader-member}, address: akka://test 01:49:01,269 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer leader-member-shard-default-config32 with address akka://test/user/shardmanager-config32/leader-member-shard-default-config32 01:49:01,269 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer leader-member-shard-default-config32 with address akka://test/user/shardmanager-config32/leader-member-shard-default-config32 to akka://test/user/member-1-shard-default-config 01:49:01,269 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config32: Received role changed for member-1-shard-default-config32 from Candidate to Follower 01:49:01,269 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config32: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config32, leaderId=leader-member-shard-default-config32, leaderPayloadVersion=13] 01:49:01,269 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config32: All Shards are ready - data store config32 is ready 01:49:01,269 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config32: onAddShardReplica: AddShardReplica[ShardName=default] 01:49:01,269 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config32: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 01:49:01,269 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config32: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://test/user/shardmanager-config32/leader-member-shard-default-config32, primaryVersion=13] 01:49:01,269 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config32: Sending AddServer message to peer akka://test/user/shardmanager-config32/leader-member-shard-default-config32 for shard member-1-shard-default-config32 01:49:01,270 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config32: Received AddServerReply{status=ALREADY_EXISTS} for shard default from leader akka://test/user/shardmanager-config32/leader-member-shard-default-config32 01:49:01,270 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config32: Local shard default already exists 01:49:01,319 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config32: findLocalShard : default 01:49:01,319 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config32: onAddShardReplica: AddShardReplica[ShardName=default] 01:49:01,320 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config32: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 01:49:01,320 PM [test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config32: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://test/user/shardmanager-config32/leader-member-shard-default-config32, primaryVersion=13] 01:49:01,320 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config32: Sending AddServer message to peer akka://test/user/shardmanager-config32/leader-member-shard-default-config32 for shard member-1-shard-default-config32 01:49:01,320 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config32: Received AddServerReply{status=ALREADY_EXISTS} for shard default from leader akka://test/user/shardmanager-config32/leader-member-shard-default-config32 01:49:01,320 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config32: Local shard default already exists 01:49:01,321 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-default-config32 01:49:01,321 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config32: onAddShardReplica: AddShardReplica[ShardName=default] 01:49:01,322 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config32: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 01:49:01,322 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config32: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://test/user/shardmanager-config32/leader-member-shard-default-config32, primaryVersion=13] 01:49:01,322 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config32: Sending AddServer message to peer akka://test/user/shardmanager-config32/leader-member-shard-default-config32 for shard member-1-shard-default-config32 01:49:01,434 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config32: AddServer request to akka://test/user/shardmanager-config32/leader-member-shard-default-config32 for default failed akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://test/), Path(/user/shardmanager-config32/leader-member-shard-default-config32)]] after [100 ms]. Message of type [org.opendaylight.controller.cluster.raft.messages.AddServer]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:49:01,435 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config32: findLocalShard : default 01:49:01,435 PM [main] [INFO] ShardManagerTest - testAddShardReplicaWithPreExistingReplicaInRemoteShardLeader ending 01:49:01,537 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/shardmanager-config32] 01:49:01,548 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Stopping ShardManager shard-manager-config32 01:49:01,550 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForRemoteShard starting 01:49:01,561 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:01,568 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-2053529103170571603] 01:49:01,569 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:49:01,571 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:49:01,571 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:49:01,580 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:01,581 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:01,584 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:49:01,584 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:49:01,584 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:49:01,588 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config33] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:01,592 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config33 01:49:01,607 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config33 01:49:01,607 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config33: Creating local shard: member-1-shard-default-config33 01:49:01,608 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config33: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:01,608 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-default-config33 01:49:01,613 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:01,655 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [4659536195248952304] 01:49:01,655 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:49:01,656 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:01,656 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:49:01,662 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:01,662 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:01,668 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config33] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:01,670 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config33 01:49:01,671 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shard-manager-config33 01:49:01,671 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config33: Creating local shard: member-2-shard-default-config33 01:49:01,671 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config33: Creating local shard: member-2-shard-astronauts-config33 01:49:01,676 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:49:01,676 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config33 01:49:01,676 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-default-config33 01:49:01,676 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-default-config33 to akka://test/user/member-1-shard-default-config 01:49:01,676 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config33: updated peer member-1-shard-default-config33 on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-default-config33 on shard member-1-shard-default-config33 whose actor address is Actor[akka://test/user/member-1-shard-default-config#-583030836] 01:49:01,676 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:49:01,676 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-2-shard-default-config33 01:49:01,676 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-2-shard-astronauts-config33 01:49:01,676 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config33: Initializing shard [astronauts] 01:49:01,676 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-2-shard-astronauts-config33 is initialized 01:49:01,676 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config33: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-astronauts-config33, leaderId=member-2-shard-astronauts-config33, leaderPayloadVersion=12] 01:49:01,676 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config33: Received role changed for member-2-shard-astronauts-config33 from Candidate to Leader 01:49:01,702 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:49:01,710 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:01,711 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config33: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:01,711 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-default-config33 01:49:01,711 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-default-config33 to akka://cluster-test/user/member-2-shard-astronauts-config 01:49:01,711 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-astronauts-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-astronauts-config33 01:49:01,711 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-astronauts-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-astronauts-config33 to akka://cluster-test/user/member-2-shard-astronauts-config 01:49:02,596 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:49:02,596 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config33: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:02,596 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config33 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33/member-2-shard-default-config33 01:49:02,596 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config33 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33/member-2-shard-default-config33 to akka://test/user/member-1-shard-default-config 01:49:02,597 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config33: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=astronauts, waitUntilReady=false] 01:49:02,597 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config33: findPrimary for astronauts forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33, visitedAddresses: [akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33] 01:49:02,606 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config33: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=astronauts, waitUntilReady=false] 01:49:02,606 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config33: Found primary for astronauts: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/member-2-shard-astronauts-config#295809431, primaryVersion=12] 01:49:02,608 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForRemoteShard ending 01:49:02,683 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config33: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:02,683 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config33 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33/member-2-shard-default-config33 01:49:02,683 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config33 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33/member-2-shard-default-config33 to akka://cluster-test/user/member-2-shard-astronauts-config 01:49:02,683 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-astronauts-config33 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33/member-2-shard-astronauts-config33 01:49:02,683 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-astronauts-config33 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33/member-2-shard-astronauts-config33 to akka://cluster-test/user/member-2-shard-astronauts-config 01:49:02,710 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-config33 01:49:02,710 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:02,710 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:02,723 PM [cluster-test-akka.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:02,723 PM [cluster-test-akka.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:02,725 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:49:02,758 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config33 01:49:02,761 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:02,761 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:03,778 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:03.780] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:03,788 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Starting ShardManager shard-manager-config34 01:49:03,789 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config34 01:49:03,789 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config34: Creating local shard: member-1-shard-default-config34 01:49:03,799 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config34: findLocalShard : default 01:49:03,799 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config34: Returning NotInitializedException for shard default 01:49:03,900 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$r#1944526076] 01:49:03,911 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config34 01:49:03,916 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Starting ShardManager shard-manager-config35 01:49:03,918 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config35 01:49:03,918 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config35: Creating local shard: member-1-shard-default-config35 01:49:03,925 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:49:03,925 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config35 01:49:03,926 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config35: Initializing shard [default] 01:49:03,926 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config35 is initialized 01:49:03,926 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config35: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config35, leaderId=member-1-shard-default-config35, leaderPayloadVersion=13] 01:49:03,926 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config35: Received role changed for member-1-shard-default-config35 from Candidate to Leader 01:49:03,926 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config35: All Shards are ready - data store config35 is ready 01:49:03,926 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config35: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=default, meberVotingStatusMap={member-2=true}] 01:49:03,926 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config35: findLocalShard : default 01:49:03,926 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config35: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-default-config35=true}, serversVisited=[]] to local shard akka://test/user/member-1-shard-default-config35 01:49:03,927 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config35: Received ServerChangeReply{status=OK} from local shard akka://test/user/member-1-shard-default-config35 01:49:03,927 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config35: ChangeServersVotingStatus succeeded for shard default 01:49:04,77 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-default-config35#891782682] 01:49:04,89 PM [main] [INFO] ShardManagerTest - testShutDown starting 01:49:04,92 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Starting ShardManager shard-manager-config36 01:49:04,93 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config36 01:49:04,93 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config36: Creating local shard: member-1-shard-shard2-config36 01:49:04,93 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config36: Creating local shard: member-1-shard-shard1-config36 01:49:04,102 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:49:04,102 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-shard2-config36 01:49:04,102 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-shard1-config36 01:49:04,102 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config36: Initializing shard [shard1] 01:49:04,102 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-shard1-config36 is initialized 01:49:04,103 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config36: Initializing shard [shard2] 01:49:04,103 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-shard2-config36 is initialized 01:49:04,103 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config36: Issuing gracefulStop to shard member-1-shard-shard2-config36 01:49:04,104 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config36: Issuing gracefulStop to shard member-1-shard-shard1-config36 01:49:04,104 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Shutting down ShardManager shard-manager-config36 - waiting on 2 shards 01:49:04,654 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-shard1-config36#-1449901657] 01:49:04,655 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-shard2-config36#1976069168] 01:49:04,656 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config36: All shards shutdown - sending PoisonPill to self 01:49:04,656 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config36 01:49:04,656 PM [main] [INFO] ShardManagerTest - testShutDown ending 01:49:04,757 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$t#-124160265] 01:49:04,770 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForNonLocalLeaderShardBeforeMemberUp starting 01:49:04,773 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config37 01:49:04,775 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config37 01:49:04,775 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config37: Creating local shard: member-1-shard-default-config37 01:49:04,784 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:49:04,784 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config37 01:49:04,784 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config37: Initializing shard [default] 01:49:04,784 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config37 is initialized 01:49:04,784 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config37: Received role changed for member-1-shard-default-config37 from Candidate to Follower 01:49:04,784 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManagerTest$TestShardManager - Received unhandled message LeaderStateChanged [memberId=member-1-shard-default-config37, leaderId=member-2-shard-default-config37, leaderPayloadVersion=13] 01:49:04,785 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config37: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 01:49:04,785 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config37: Returning NoShardLeaderException for shard default 01:49:04,785 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForNonLocalLeaderShardBeforeMemberUp ending 01:49:04,886 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$u#1789310577] 01:49:04,897 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config37 01:49:04,898 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithNoRoleShard starting 01:49:04,900 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config38 01:49:04,901 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config38 01:49:04,902 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config38: Creating local shard: member-1-shard-default-config38 01:49:04,911 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:49:04,911 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config38 01:49:04,911 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config38: Initializing shard [default] 01:49:04,911 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config38 is initialized 01:49:04,911 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config38: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 01:49:04,911 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config38: Scheduling 1200 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config38, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 01:49:06,124 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config38: Received ShardNotInitializedTimeout message for shard default 01:49:06,124 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config38: Returning NoShardLeaderException for shard default 01:49:06,124 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithNoRoleShard ending 01:49:06,226 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$v#-2059749065] 01:49:06,237 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config38 01:49:06,239 PM [main] [INFO] ShardManagerTest - testShardPersistenceWithRestoredData starting 01:49:06,242 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config39 01:49:06,244 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config39: onSnapshotOffer: ShardManagerSnapshot [ShardList = [default, astronauts] ] 01:49:06,244 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config39: adding shard astronauts 01:49:06,244 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config39: removing shard people 01:49:06,245 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config39 01:49:06,245 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config39: Creating local shard: member-1-shard-default-config39 01:49:06,245 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config39: Creating local shard: member-1-shard-astronauts-config39 01:49:06,255 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config39: findLocalShard : people 01:49:06,255 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config39: Local shard people not found - shards present: [default, astronauts] 01:49:06,256 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config39: findLocalShard : default 01:49:06,256 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config39: Returning NotInitializedException for shard default 01:49:06,256 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config39: findLocalShard : astronauts 01:49:06,256 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config39: Returning NotInitializedException for shard astronauts 01:49:06,256 PM [main] [INFO] ShardManagerTest - testShardPersistenceWithRestoredData ending 01:49:06,358 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$o] 01:49:06,369 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config39 01:49:06,370 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForNonLocalLeaderShard starting 01:49:06,372 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config40 01:49:06,373 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config40 01:49:06,374 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config40: Creating local shard: member-1-shard-default-config40 01:49:06,383 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:49:06,383 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config40 01:49:06,383 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config40: Initializing shard [default] 01:49:06,383 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config40 is initialized 01:49:06,383 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config40: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://test 01:49:06,383 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config40 with address akka://test/user/shardmanager-config40/member-2-shard-default-config40 01:49:06,383 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config40 with address akka://test/user/shardmanager-config40/member-2-shard-default-config40 to akka://test/user/member-1-shard-default-config 01:49:06,383 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config40: Received role changed for member-1-shard-default-config40 from Candidate to Follower 01:49:06,383 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config40: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config40, leaderId=member-2-shard-default-config40, leaderPayloadVersion=12] 01:49:06,383 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config40: All Shards are ready - data store config40 is ready 01:49:06,384 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config40: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 01:49:06,384 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config40: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://test/user/shardmanager-config40/member-2-shard-default-config40, primaryVersion=12] 01:49:06,384 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForNonLocalLeaderShard ending 01:49:06,485 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$w#558126542] 01:49:06,496 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config40 01:49:06,498 PM [main] [INFO] ShardManagerTest - testShardAvailabilityOnChangeOfMemberReachability starting 01:49:06,505 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:06,514 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [5224459675314047464] 01:49:06,514 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:49:06,515 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:49:06,515 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:49:06,516 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:06,516 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:06,527 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:49:06,527 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:49:06,527 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:49:06,527 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config41] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:06,533 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config41 01:49:06,541 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config41 01:49:06,541 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config41: Creating local shard: member-1-shard-default-config41 01:49:06,542 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config41: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:06,542 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-1-shard-default-config41 01:49:06,543 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:06,559 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-2102762567453819551] 01:49:06,559 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:49:06,561 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:06,561 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:49:06,568 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:06,568 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:06,577 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config41 01:49:06,578 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config41] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:06,583 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shard-manager-config41 01:49:06,583 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config41: Creating local shard: member-2-shard-default-config41 01:49:06,584 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:49:06,584 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config41 01:49:06,584 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-1-shard-default-config41 01:49:06,584 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-1-shard-default-config41 to akka://cluster-test/user/member-1-shard-default-config 01:49:06,584 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config41: updated peer member-1-shard-default-config41 on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-1-shard-default-config41 on shard member-1-shard-default-config41 whose actor address is Actor[akka://cluster-test/user/member-1-shard-default-config#1610989337] 01:49:06,584 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config41: Initializing shard [default] 01:49:06,584 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-default-config41 is initialized 01:49:06,584 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config41: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config41, leaderId=member-2-shard-default-config41, leaderPayloadVersion=13] 01:49:06,584 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config41: Received role changed for member-1-shard-default-config41 from Candidate to Follower 01:49:06,584 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:49:06,584 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-2-shard-default-config41 01:49:06,584 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config41: Initializing shard [default] 01:49:06,584 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardInformation - Shard member-2-shard-default-config41 is initialized 01:49:06,584 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config41: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-config41, leaderId=member-2-shard-default-config41, leaderPayloadVersion=13] 01:49:06,584 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config41: Received role changed for member-2-shard-default-config41 from Candidate to Leader 01:49:06,584 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config41: All Shards are ready - data store config41 is ready 01:49:06,619 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:49:06,625 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:06,626 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config41: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:06,626 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-1-shard-default-config41 01:49:06,626 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-1-shard-default-config41 to akka://cluster-test/user/member-2-shard-default-config 01:49:06,626 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config41: All Shards are ready - data store config41 is ready 01:49:07,534 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:49:07,535 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config41: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:07,535 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config41/member-2-shard-default-config41 01:49:07,535 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config41/member-2-shard-default-config41 to akka://cluster-test/user/member-1-shard-default-config 01:49:07,535 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config41: All Shards are ready - data store config41 is ready 01:49:07,536 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config41: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 01:49:07,536 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config41: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config41/member-2-shard-default-config41, primaryVersion=13] 01:49:07,536 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 01:49:07,536 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Marking Leader member-2-shard-default-config41 as unavailable. 01:49:07,537 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config41: Received MemberRemoved: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 01:49:07,537 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config41: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 01:49:07,537 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config41: Scheduling 1200 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config41, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=member-2-shard-default-config41, activeMember=true] 01:49:07,594 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config41: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:07,595 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config41/member-2-shard-default-config41 01:49:07,595 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config41/member-2-shard-default-config41 to akka://cluster-test/user/member-2-shard-default-config 01:49:07,595 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config41: All Shards are ready - data store config41 is ready 01:49:08,751 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config41: Received ShardNotInitializedTimeout message for shard default 01:49:08,751 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config41: Returning NoShardLeaderException for shard default 01:49:08,752 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Received ReachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 01:49:08,752 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41 01:49:08,752 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41 to akka://cluster-test/user/member-1-shard-default-config 01:49:08,752 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Marking Leader member-2-shard-default-config41 as available. 01:49:08,753 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config41: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 01:49:08,753 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config41: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41, primaryVersion=13] 01:49:08,753 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config41: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 01:49:08,753 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41 01:49:08,753 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41 to akka://cluster-test/user/member-1-shard-default-config 01:49:08,753 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config41: All Shards are ready - data store config41 is ready 01:49:08,753 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 01:49:08,753 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Marking Leader member-2-shard-default-config41 as unavailable. 01:49:08,753 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config41: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 01:49:08,753 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config41: Scheduling 1200 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config41, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=member-2-shard-default-config41, activeMember=true] 01:49:08,754 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Received ReachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 01:49:08,754 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41 01:49:08,754 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41 to akka://cluster-test/user/member-1-shard-default-config 01:49:08,754 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-default-config41 is initialized - notifying 1 OnShardInitialized callbacks 01:49:08,754 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Marking Leader member-2-shard-default-config41 as available. 01:49:08,754 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-default-config41 is ready - notifying 1 OnShardInitialized callbacks 01:49:08,754 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config41: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41, primaryVersion=13] 01:49:08,754 PM [main] [INFO] ShardManagerTest - testShardAvailabilityOnChangeOfMemberReachability ending 01:49:08,856 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config41 01:49:08,856 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:08,856 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:08,864 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:08,864 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:08,867 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:49:08,889 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config41 01:49:08,892 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:08,892 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:09,909 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:09.911] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:09,932 PM [main] [INFO] ShardManagerTest - testAddShardReplica starting 01:49:09,940 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:09,949 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [5248860223420765076] 01:49:09,949 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:49:09,959 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:49:09,959 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:49:09,961 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:09,961 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:09,974 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:49:09,977 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:49:09,977 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:49:09,979 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shardmanager-config42 01:49:09,979 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config42] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:09,980 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:09,987 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-5008436066628200263] 01:49:09,987 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:49:09,989 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:09,989 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:49:09,999 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:10,0 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:10,0 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shardmanager-config42 01:49:10,0 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shardmanager-config42: Creating local shard: member-1-shard-default-config42 01:49:10,0 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shardmanager-config42: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:10,0 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-default-config42 01:49:10,4 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shardmanager-config42 01:49:10,6 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config42] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:10,10 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:49:10,10 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config42 01:49:10,10 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-default-config42 01:49:10,10 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-default-config42 to akka://cluster-test/user/member-1-shard-default-config 01:49:10,10 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shardmanager-config42: updated peer member-1-shard-default-config42 on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-default-config42 on shard member-1-shard-default-config42 whose actor address is Actor[akka://cluster-test/user/member-1-shard-default-config#-1282529737] 01:49:10,13 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shardmanager-config42 01:49:10,13 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shardmanager-config42: Creating local shard: member-2-shard-default-config42 01:49:10,13 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shardmanager-config42: Creating local shard: member-2-shard-astronauts-config42 01:49:10,13 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:49:10,14 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-2-shard-default-config42 01:49:10,14 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-2-shard-astronauts-config42 01:49:10,14 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shardmanager-config42: Initializing shard [astronauts] 01:49:10,14 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-2-shard-astronauts-config42 is initialized 01:49:10,14 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shardmanager-config42: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-astronauts-config42, leaderId=member-2-shard-astronauts-config42, leaderPayloadVersion=12] 01:49:10,14 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shardmanager-config42: Received role changed for member-2-shard-astronauts-config42 from Candidate to Leader 01:49:10,86 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:49:10,90 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:10,91 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shardmanager-config42: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:10,91 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-default-config42 01:49:10,91 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-default-config42 to akka://cluster-test/user/member-2-shard-astronauts-config 01:49:10,91 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-astronauts-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-astronauts-config42 01:49:10,91 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-astronauts-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-astronauts-config42 to akka://cluster-test/user/member-2-shard-astronauts-config 01:49:10,980 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:49:10,980 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shardmanager-config42: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:10,980 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config42 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42/member-2-shard-default-config42 01:49:10,980 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config42 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42/member-2-shard-default-config42 to akka://cluster-test/user/member-1-shard-default-config 01:49:10,983 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shardmanager-config42: onAddShardReplica: AddShardReplica[ShardName=astronauts] 01:49:10,983 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shardmanager-config42: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=astronauts, waitUntilReady=true] 01:49:10,983 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shardmanager-config42: findPrimary for astronauts forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42, visitedAddresses: [akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42] 01:49:10,986 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shardmanager-config42: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=astronauts, waitUntilReady=true] 01:49:10,986 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shardmanager-config42: Found primary for astronauts: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/member-2-shard-astronauts-config, primaryVersion=12] 01:49:10,990 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shardmanager-config42: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2559/user/member-2-shard-astronauts-config for shard member-1-shard-astronauts-config42 01:49:10,996 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shardmanager-config42: Received AddServerReply{status=OK, leaderHint=member-2-shard-astronauts-config42} for shard astronauts from leader akka://cluster-test@127.0.0.1:2559/user/member-2-shard-astronauts-config 01:49:10,996 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shardmanager-config42: Leader shard successfully added the replica shard astronauts 01:49:10,997 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-astronauts-config42 01:49:10,997 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shardmanager-config42: persisting the shard list [default, astronauts] 01:49:10,997 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shardmanager-config42 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 01:49:10,997 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shardmanager-config42: Successfully deleted prior snapshots 01:49:11,26 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shardmanager-config42: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:11,26 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config42 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42/member-2-shard-default-config42 01:49:11,26 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config42 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42/member-2-shard-default-config42 to akka://cluster-test/user/member-2-shard-astronauts-config 01:49:11,26 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-astronauts-config42 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42/member-2-shard-astronauts-config42 01:49:11,26 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-astronauts-config42 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42/member-2-shard-astronauts-config42 to akka://cluster-test/user/member-2-shard-astronauts-config 01:49:11,33 PM [main] [INFO] ShardManagerTest - testAddShardReplica ending 01:49:11,135 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shardmanager-config42 01:49:11,136 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:11,136 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:11,148 PM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:11,148 PM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:11,150 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:49:11,158 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shardmanager-config42 01:49:11,159 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:11,159 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:12,179 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:12.181] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:12,208 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Starting ShardManager shard-manager-config43 01:49:12,209 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config43 01:49:12,209 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config43: Creating local shard: member-1-shard-default-config43 01:49:12,217 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:49:12,217 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config43 01:49:12,217 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config43: Initializing shard [default] 01:49:12,217 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-default-config43 is initialized 01:49:12,217 PM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - shard-manager-config43: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config43, leaderId=member-1-shard-default-config43, leaderPayloadVersion=13] 01:49:12,217 PM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - shard-manager-config43: Received role changed for member-1-shard-default-config43 from Candidate to Leader 01:49:12,217 PM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - shard-manager-config43: All Shards are ready - data store config43 is ready 01:49:12,217 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config43: onRemoveShardReplica: RemoveShardReplica [shardName=default, memberName=MemberName{name=member-1}] 01:49:12,217 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config43: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 01:49:12,218 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config43: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://test@127.0.0.1:2565/user/member-1-shard-default-config43#1124883563, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$qaTj6l0k@558c87b] 01:49:12,218 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config43: Sending RemoveServer message to peer akka://test@127.0.0.1:2565/user/member-1-shard-default-config43#1124883563 for shard member-1-shard-default-config43 01:49:12,218 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config43: Received RemoveServerReply{status=OK} for shard default 01:49:12,218 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config43: Leader shard successfully removed the replica shard default 01:49:12,370 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-default-config43#1124883563] 01:49:12,383 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithIsolatedLeaderShard starting 01:49:12,385 PM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-config44 01:49:12,387 PM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-config44 01:49:12,387 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config44: Creating local shard: member-1-shard-default-config44 01:49:12,396 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:49:12,396 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config44 01:49:12,396 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config44: Initializing shard [default] 01:49:12,396 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config44 is initialized 01:49:12,396 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config44: Received role changed for member-1-shard-default-config44 from null to IsolatedLeader 01:49:12,396 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config44: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 01:49:12,396 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config44: Scheduling 1200 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config44, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=null, activeMember=true] 01:49:13,614 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config44: Received ShardNotInitializedTimeout message for shard default 01:49:13,614 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config44: Returning NoShardLeaderException for shard default 01:49:13,614 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithIsolatedLeaderShard ending 01:49:13,716 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$y#376512135] 01:49:13,727 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config44 01:49:13,731 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config45 01:49:13,732 PM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-config45 01:49:13,732 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config45: Creating local shard: member-1-shard-default-config45 01:49:13,741 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:49:13,741 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config45 01:49:13,741 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config45: findLocalShard : non-existent 01:49:13,741 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config45: Local shard non-existent not found - shards present: [default] 01:49:13,843 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$z#1346517116] 01:49:13,854 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config45 01:49:13,858 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config46 01:49:13,859 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config46 01:49:13,879 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:49:13,879 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config46: onRemoveShardReplica: RemoveShardReplica [shardName=astronauts, memberName=MemberName{name=member-2}] 01:49:13,879 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config46: Sending RemoveServer message to peer akka://test@127.0.0.1:2565/system/testProbe-617#220876177 for shard member-2-shard-astronauts-config46 01:49:13,890 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config46: onRemoveShardReplica: RemoveShardReplica [shardName=astronauts, memberName=MemberName{name=member-3}] 01:49:13,890 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config46: A shard replica operation for astronauts is already in progress 01:49:13,997 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config47 01:49:13,998 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config47 01:49:13,998 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config47: Creating local shard: member-1-shard-default-config47 01:49:14,7 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config47: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 01:49:14,7 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config47: Returning NotInitializedException for shard default 01:49:14,109 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$A#1988405494] 01:49:14,119 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config47 01:49:14,121 PM [main] [INFO] ShardManagerTest - testServerRemovedShardActorRunning starting 01:49:14,123 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config48 01:49:14,125 PM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-config48 01:49:14,125 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config48: Creating local shard: member-1-shard-default-config48 01:49:14,125 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config48: Creating local shard: member-1-shard-people-config48 01:49:14,136 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:49:14,136 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config48 01:49:14,136 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config48 01:49:14,136 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config48: Initializing shard [default] 01:49:14,136 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config48 is initialized 01:49:14,136 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config48: findLocalShard : people 01:49:14,136 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config48: Scheduling 600 ms timer to wait for shard ShardInformation [shardId=member-1-shard-people-config48, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 01:49:14,138 PM [test-shard-dispatcher-17] [INFO] Shard - Shard created : member-1-shard-people-config48, persistent : true 01:49:14,141 PM [test-shard-dispatcher-17] [INFO] Shard - Starting recovery for member-1-shard-people-config48 with journal batch size 1 01:49:14,142 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$p/member-1-shard-people-config48/shard-member-1:snapshot-read#1184257284] 01:49:14,142 PM [test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$p/member-1-shard-people-config48/member-1-shard-people-config48-notifier#-413372044 created and ready for shard:member-1-shard-people-config48 01:49:14,143 PM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config48: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 01:49:14,143 PM [test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config48: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:14,143 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config48: Initializing shard [people] 01:49:14,143 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-people-config48 is initialized 01:49:14,143 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-people-config48 is initialized - notifying 1 OnShardInitialized callbacks 01:49:14,143 PM [test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config48: onLeadershipChange, isLeader: false, hasLeader : false 01:49:14,143 PM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config48: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config48 is no longer the leader 01:49:14,143 PM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config48 , received role change from null to Follower 01:49:14,143 PM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config48 , registered listener akka://test/user/$$p 01:49:14,143 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config48: findLocalShard : default 01:49:14,144 PM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - shard-manager-config48: Received role changed for member-1-shard-people-config48 from null to Follower 01:49:14,144 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config48 : Sending Shutdown to Shard actor Actor[akka://test/user/member-1-shard-default-config48#-1636532255] with 10000 ms timeout 01:49:14,147 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config48 : Local Shard replica for shard default has been removed 01:49:14,147 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config48: persisting the shard list [people] 01:49:14,147 PM [main] [INFO] ShardManagerTest - testServerRemovedShardActorRunning ending 01:49:14,147 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config48 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 01:49:14,147 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config48: Successfully deleted prior snapshots 01:49:14,248 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-default-config48#-1636532255] 01:49:14,259 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config48 : Successfully shut down Shard actor Actor[akka://test/user/member-1-shard-default-config48#-1636532255] 01:49:14,259 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$p] 01:49:14,260 PM [test-shard-dispatcher-17] [INFO] Shard - Stopping Shard member-1-shard-people-config48 01:49:14,260 PM [test-shard-dispatcher-17] [DEBUG] MessageAssembler - member-1-shard-people-config48 (Follower): Closing 01:49:14,260 PM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config48: Aborting 0 pending queued transactions 01:49:14,261 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Stopping ShardManager shard-manager-config48 01:49:14,265 PM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config49 01:49:14,268 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config49 01:49:14,268 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config49: Creating local shard: member-1-shard-default-config49 01:49:14,268 PM [main] [INFO] ShardManager - shard-manager-config49: Received role changed for member-1-shard-default-config49 from null to Follower 01:49:14,269 PM [main] [INFO] ShardManager - shard-manager-config49: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config49, leaderId=member-2-shard-default-config49, leaderPayloadVersion=13] 01:49:14,279 PM [main] [INFO] ShardManager - shard-manager-config49: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://test 01:49:14,279 PM [main] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config49 with address akka://test/user/shardmanager-config49/member-2-shard-default-config49 01:49:14,279 PM [main] [INFO] ShardManager - shard-manager-config49: All Shards are ready - data store config49 is ready 01:49:14,380 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$q] 01:49:14,391 PM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config49 01:49:14,392 PM [main] [INFO] ShardManagerTest - testRestoreFromSnapshot starting 01:49:14,395 PM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-config50 01:49:14,396 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config50 01:49:14,396 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config50: Restoring from ShardManagerSnapshot: ShardManagerSnapshot [ShardList = [shard1, shard2, astronauts] ] 01:49:14,396 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config50: onSnapshotOffer: ShardManagerSnapshot [ShardList = [shard1, shard2, astronauts] ] 01:49:14,396 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config50: adding shard shard1 01:49:14,396 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config50: adding shard shard2 01:49:14,396 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config50: adding shard astronauts 01:49:14,396 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config50: Creating local shard: member-1-shard-shard2-config50 01:49:14,396 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config50: Creating local shard: member-1-shard-astronauts-config50 01:49:14,396 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config50: Creating local shard: member-1-shard-shard1-config50 01:49:14,397 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:49:14,397 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-shard2-config50 01:49:14,397 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-astronauts-config50 01:49:14,398 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-shard1-config50 01:49:14,398 PM [test-shard-dispatcher-17] [INFO] Shard - Shard created : member-1-shard-shard2-config50, persistent : true 01:49:14,398 PM [test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-astronauts-config50, persistent : true 01:49:14,399 PM [test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-astronauts-config50 with journal batch size 1 01:49:14,399 PM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$r/member-1-shard-astronauts-config50/member-1-shard-astronauts-config50-notifier#-943642296 created and ready for shard:member-1-shard-astronauts-config50 01:49:14,399 PM [test-shard-dispatcher-17] [INFO] Shard - Starting recovery for member-1-shard-shard2-config50 with journal batch size 1 01:49:14,399 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$r/member-1-shard-astronauts-config50/shard-member-1:snapshot-read#-9852399] 01:49:14,400 PM [test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-shard1-config50, persistent : true 01:49:14,400 PM [test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$r/member-1-shard-shard1-config50/member-1-shard-shard1-config50-notifier#1656634853 created and ready for shard:member-1-shard-shard1-config50 01:49:14,400 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$r/member-1-shard-shard2-config50/shard-member-1:snapshot-read#1345163465] 01:49:14,401 PM [test-akka.actor.default-dispatcher-20] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$r/member-1-shard-shard1-config50/shard-member-1:snapshot-read#-143605819] 01:49:14,401 PM [test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-shard1-config50 with journal batch size 1 01:49:14,401 PM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-shard2-config50: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 01:49:14,401 PM [test-shard-dispatcher-17] [INFO] Shard - member-1-shard-shard2-config50: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:14,401 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config50: Initializing shard [shard2] 01:49:14,401 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-shard2-config50 is initialized 01:49:14,401 PM [test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard2-config50: onLeadershipChange, isLeader: false, hasLeader : false 01:49:14,401 PM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-shard2-config50: onStateChanged: Closing all transaction chains because shard member-1-shard-shard2-config50 is no longer the leader 01:49:14,402 PM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-shard2-config50 (Follower): Received TimeoutNow - switching to Candidate 01:49:14,402 PM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-shard2-config50 (Candidate): Election: Candidate has following voting peers: [] 01:49:14,402 PM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-shard2-config50: Set currentTerm=1, votedFor=member-1-shard-shard2-config50 01:49:14,402 PM [test-shard-dispatcher-17] [INFO] Shard - member-1-shard-shard2-config50 (Candidate): Starting new election term 1 01:49:14,402 PM [test-shard-dispatcher-17] [INFO] Shard - member-1-shard-shard2-config50 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:49:14,402 PM [test-shard-dispatcher-17] [DEBUG] MessageAssembler - member-1-shard-shard2-config50 (Follower): Closing 01:49:14,402 PM [test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard2-config50: onLeadershipChange, isLeader: false, hasLeader : false 01:49:14,402 PM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-shard2-config50: onStateChanged: Closing all transaction chains because shard member-1-shard-shard2-config50 is no longer the leader 01:49:14,403 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard1-config50: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 01:49:14,403 PM [test-shard-dispatcher-19] [INFO] Shard - member-1-shard-shard1-config50: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:14,403 PM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-shard2-config50 (Candidate): Received ElectionTimeout 01:49:14,403 PM [test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard1-config50: onLeadershipChange, isLeader: false, hasLeader : false 01:49:14,403 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard1-config50: onStateChanged: Closing all transaction chains because shard member-1-shard-shard1-config50 is no longer the leader 01:49:14,403 PM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-shard2-config50 (Leader): Election: Leader has following peers: [] 01:49:14,403 PM [test-shard-dispatcher-17] [INFO] Shard - member-1-shard-shard2-config50 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 01:49:14,403 PM [test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard2-config50: onLeadershipChange, isLeader: true, hasLeader : true 01:49:14,403 PM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-shard2-config50: became leader with frontend state for [] 01:49:14,403 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard1-config50 (Follower): Received TimeoutNow - switching to Candidate 01:49:14,403 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard1-config50 (Candidate): Election: Candidate has following voting peers: [] 01:49:14,403 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard1-config50: Set currentTerm=1, votedFor=member-1-shard-shard1-config50 01:49:14,403 PM [test-shard-dispatcher-19] [INFO] Shard - member-1-shard-shard1-config50 (Candidate): Starting new election term 1 01:49:14,403 PM [test-shard-dispatcher-19] [INFO] Shard - member-1-shard-shard1-config50 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:49:14,403 PM [test-shard-dispatcher-19] [DEBUG] MessageAssembler - member-1-shard-shard1-config50 (Follower): Closing 01:49:14,403 PM [test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard1-config50: onLeadershipChange, isLeader: false, hasLeader : false 01:49:14,403 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard1-config50: onStateChanged: Closing all transaction chains because shard member-1-shard-shard1-config50 is no longer the leader 01:49:14,403 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config50: Initializing shard [shard1] 01:49:14,403 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-shard1-config50 is initialized 01:49:14,403 PM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config50 , received role change from null to Follower 01:49:14,403 PM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config50 , received role change from Follower to Candidate 01:49:14,404 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard1-config50 (Candidate): Received ElectionTimeout 01:49:14,404 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard1-config50 (Leader): Election: Leader has following peers: [] 01:49:14,404 PM [test-shard-dispatcher-19] [INFO] Shard - member-1-shard-shard1-config50 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 01:49:14,404 PM [test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard1-config50: onLeadershipChange, isLeader: true, hasLeader : true 01:49:14,404 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard1-config50: became leader with frontend state for [] 01:49:14,404 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 01:49:14,404 PM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-astronauts-config50: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:14,404 PM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config50: onLeadershipChange, isLeader: false, hasLeader : false 01:49:14,404 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50: onStateChanged: Closing all transaction chains because shard member-1-shard-astronauts-config50 is no longer the leader 01:49:14,404 PM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config50 , received role change from Candidate to Leader 01:49:14,404 PM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config50 , registered listener akka://test/user/$$r 01:49:14,404 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50 (Follower): Received TimeoutNow - switching to Candidate 01:49:14,404 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config50: Initializing shard [astronauts] 01:49:14,404 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-astronauts-config50 is initialized 01:49:14,404 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50 (Candidate): Election: Candidate has following voting peers: [] 01:49:14,404 PM [test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config50 , received role change from null to Follower 01:49:14,404 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50: Set currentTerm=1, votedFor=member-1-shard-astronauts-config50 01:49:14,404 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config50: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-shard1-config50, leaderId=member-1-shard-shard1-config50, leaderPayloadVersion=13] 01:49:14,404 PM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-astronauts-config50 (Candidate): Starting new election term 1 01:49:14,404 PM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-astronauts-config50 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:49:14,404 PM [test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-1-shard-astronauts-config50 (Follower): Closing 01:49:14,404 PM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config50: onLeadershipChange, isLeader: false, hasLeader : false 01:49:14,404 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config50: Received role changed for member-1-shard-shard1-config50 from Candidate to Leader 01:49:14,404 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50: onStateChanged: Closing all transaction chains because shard member-1-shard-astronauts-config50 is no longer the leader 01:49:14,404 PM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config50 , received role change from Follower to Candidate 01:49:14,405 PM [test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$r/member-1-shard-shard2-config50/member-1-shard-shard2-config50-notifier#-970672059 created and ready for shard:member-1-shard-shard2-config50 01:49:14,405 PM [test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config50 , received role change from null to Follower 01:49:14,405 PM [test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config50 , received role change from Follower to Candidate 01:49:14,405 PM [test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config50 , received role change from Candidate to Leader 01:49:14,405 PM [test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config50 , registered listener akka://test/user/$$r 01:49:14,405 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config50: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-shard2-config50, leaderId=member-1-shard-shard2-config50, leaderPayloadVersion=13] 01:49:14,405 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config50: Received role changed for member-1-shard-shard2-config50 from Candidate to Leader 01:49:14,405 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50 (Candidate): Received ElectionTimeout 01:49:14,405 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50 (Leader): Election: Leader has following peers: [] 01:49:14,405 PM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-astronauts-config50 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 01:49:14,405 PM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config50: onLeadershipChange, isLeader: true, hasLeader : true 01:49:14,405 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50: became leader with frontend state for [] 01:49:14,405 PM [test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config50 , received role change from Candidate to Leader 01:49:14,405 PM [test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config50 , registered listener akka://test/user/$$r 01:49:14,405 PM [test-akka.actor.default-dispatcher-23] [INFO] ShardManager - shard-manager-config50: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-astronauts-config50, leaderId=member-1-shard-astronauts-config50, leaderPayloadVersion=13] 01:49:14,405 PM [test-akka.actor.default-dispatcher-23] [INFO] ShardManager - shard-manager-config50: Received role changed for member-1-shard-astronauts-config50 from Candidate to Leader 01:49:14,405 PM [test-akka.actor.default-dispatcher-23] [INFO] ShardManager - shard-manager-config50: All Shards are ready - data store config50 is ready 01:49:14,407 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config50: findLocalShard : shard1 01:49:14,407 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config50: findLocalShard : shard2 01:49:14,407 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config50: findLocalShard : astronauts 01:49:14,408 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config50: onGetSnapshot 01:49:14,408 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config50: onGetSnapshot 01:49:14,408 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config50: Expecting 3 shard snapshot replies 01:49:14,408 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config50: onGetSnapshot 01:49:14,408 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config50: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 01:49:14,408 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config50: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 01:49:14,408 PM [test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-shard1-config50: onGetSnapshot 01:49:14,408 PM [test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-shard1-config50: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 01:49:14,412 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config50: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 01:49:14,412 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config50: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 01:49:14,412 PM [test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-shard1-config50: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 01:49:14,412 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config50: Received GetSnapshotReply [id=member-1-shard-astronauts-config50, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@36d222d5, electionTerm=1, electionVotedFor=member-1-shard-astronauts-config50, ServerConfigPayload=null]] 01:49:14,412 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config50: Received GetSnapshotReply [id=member-1-shard-shard2-config50, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@40c0a932, electionTerm=1, electionVotedFor=member-1-shard-shard2-config50, ServerConfigPayload=null]] 01:49:14,412 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config50: Received GetSnapshotReply [id=member-1-shard-shard1-config50, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@4d054fae, electionTerm=1, electionVotedFor=member-1-shard-shard1-config50, ServerConfigPayload=null]] 01:49:14,412 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config50: All shard snapshots received 01:49:14,413 PM [main] [INFO] ShardManagerTest - testRestoreFromSnapshot ending 01:49:14,514 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$r] 01:49:14,525 PM [test-shard-dispatcher-18] [INFO] Shard - Stopping Shard member-1-shard-astronauts-config50 01:49:14,525 PM [test-shard-dispatcher-19] [INFO] Shard - Stopping Shard member-1-shard-shard1-config50 01:49:14,525 PM [test-shard-dispatcher-25] [INFO] Shard - Stopping Shard member-1-shard-shard2-config50 01:49:14,525 PM [test-shard-dispatcher-19] [DEBUG] MessageSlicer - member-1-shard-shard1-config50 (Leader)_slicer-id-231: Closing 01:49:14,525 PM [test-shard-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-shard2-config50 (Leader)_slicer-id-230: Closing 01:49:14,525 PM [test-shard-dispatcher-18] [DEBUG] MessageSlicer - member-1-shard-astronauts-config50 (Leader)_slicer-id-232: Closing 01:49:14,525 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50: Aborting 0 pending queued transactions 01:49:14,525 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard1-config50: Aborting 0 pending queued transactions 01:49:14,525 PM [test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-shard2-config50: Aborting 0 pending queued transactions 01:49:14,530 PM [test-akka.actor.default-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-config50 01:49:14,532 PM [main] [INFO] ShardManagerTest - testOnCreateShardWithNoInitialSchemaContext starting 01:49:14,535 PM [test-akka.actor.default-dispatcher-23] [INFO] ShardManager - Starting ShardManager shard-manager-config51 01:49:14,536 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config51 01:49:14,546 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config51: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=foo-ns, moduleName=foo-module, shardName=foo, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@402d0a19] 01:49:14,546 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config51 doCreateShard: shardId: member-1-shard-foo-config51, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: true 01:49:14,547 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:49:14,547 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-foo-config51 01:49:14,548 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config51: findLocalShard : foo 01:49:14,548 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config51: Scheduling 600 ms timer to wait for shard ShardInformation [shardId=member-1-shard-foo-config51, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 01:49:14,548 PM [test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-foo-config51, persistent : true 01:49:14,549 PM [test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-foo-config51 with journal batch size 1 01:49:14,550 PM [test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$B/member-1-shard-foo-config51/shard-member-1:snapshot-read#165167292] 01:49:14,550 PM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$B/member-1-shard-foo-config51/member-1-shard-foo-config51-notifier#-2129870651 created and ready for shard:member-1-shard-foo-config51 01:49:14,552 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-foo-config51: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 01:49:14,552 PM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-foo-config51: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:14,552 PM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-foo-config51: onLeadershipChange, isLeader: false, hasLeader : false 01:49:14,552 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-foo-config51: onStateChanged: Closing all transaction chains because shard member-1-shard-foo-config51 is no longer the leader 01:49:14,552 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-foo-config51 (Follower): Received TimeoutNow - switching to Candidate 01:49:14,552 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-foo-config51 (Candidate): Election: Candidate has following voting peers: [] 01:49:14,552 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-foo-config51: Set currentTerm=1, votedFor=member-1-shard-foo-config51 01:49:14,552 PM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-foo-config51 (Candidate): Starting new election term 1 01:49:14,552 PM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-foo-config51 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:49:14,552 PM [test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-1-shard-foo-config51 (Follower): Closing 01:49:14,552 PM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-foo-config51: onLeadershipChange, isLeader: false, hasLeader : false 01:49:14,552 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-foo-config51: onStateChanged: Closing all transaction chains because shard member-1-shard-foo-config51 is no longer the leader 01:49:14,553 PM [test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config51 , received role change from null to Follower 01:49:14,553 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config51: Initializing shard [foo] 01:49:14,553 PM [test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config51 , received role change from Follower to Candidate 01:49:14,553 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-foo-config51 is initialized 01:49:14,553 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-foo-config51 is initialized - notifying 1 OnShardInitialized callbacks 01:49:14,553 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-foo-config51 (Candidate): Received ElectionTimeout 01:49:14,554 PM [main] [INFO] ShardManagerTest - testOnCreateShardWithNoInitialSchemaContext ending 01:49:14,554 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-foo-config51 (Leader): Election: Leader has following peers: [] 01:49:14,554 PM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-foo-config51 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 01:49:14,554 PM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-foo-config51: onLeadershipChange, isLeader: true, hasLeader : true 01:49:14,554 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-foo-config51: became leader with frontend state for [] 01:49:14,554 PM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config51 , received role change from Candidate to Leader 01:49:14,554 PM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config51 , registered listener akka://test/user/$B 01:49:14,554 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config51: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-foo-config51, leaderId=member-1-shard-foo-config51, leaderPayloadVersion=13] 01:49:14,554 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config51: Received role changed for member-1-shard-foo-config51 from Candidate to Leader 01:49:14,554 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config51: All Shards are ready - data store config51 is ready 01:49:14,655 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$B#2049350455] 01:49:14,666 PM [test-shard-dispatcher-18] [INFO] Shard - Stopping Shard member-1-shard-foo-config51 01:49:14,666 PM [test-shard-dispatcher-18] [DEBUG] MessageSlicer - member-1-shard-foo-config51 (Leader)_slicer-id-234: Closing 01:49:14,667 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-foo-config51: Aborting 0 pending queued transactions 01:49:14,667 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config51 01:49:14,668 PM [main] [INFO] ShardManagerTest - testGetSnapshot starting 01:49:14,670 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config52 01:49:14,671 PM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-config52 01:49:14,672 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config52: Creating local shard: member-1-shard-shard2-config52 01:49:14,672 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config52: Creating local shard: member-1-shard-shard1-config52 01:49:14,681 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config52: onGetSnapshot 01:49:14,681 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:49:14,681 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-shard2-config52 01:49:14,681 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-shard1-config52 01:49:14,681 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config52: findLocalShard : shard1 01:49:14,681 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config52: Scheduling 600 ms timer to wait for shard ShardInformation [shardId=member-1-shard-shard1-config52, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 01:49:14,682 PM [test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-shard2-config52, persistent : true 01:49:14,682 PM [test-shard-dispatcher-25] [INFO] Shard - Shard created : member-1-shard-shard1-config52, persistent : true 01:49:14,683 PM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$s/member-1-shard-shard2-config52/member-1-shard-shard2-config52-notifier#-587780821 created and ready for shard:member-1-shard-shard2-config52 01:49:14,683 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$s/member-1-shard-shard2-config52/shard-member-1:snapshot-read#498993453] 01:49:14,683 PM [test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-shard2-config52 with journal batch size 1 01:49:14,683 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$s/member-1-shard-shard1-config52/shard-member-1:snapshot-read#-1891233575] 01:49:14,684 PM [test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$s/member-1-shard-shard1-config52/member-1-shard-shard1-config52-notifier#-204222938 created and ready for shard:member-1-shard-shard1-config52 01:49:14,684 PM [test-shard-dispatcher-25] [INFO] Shard - Starting recovery for member-1-shard-shard1-config52 with journal batch size 1 01:49:14,684 PM [test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-shard2-config52: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 01:49:14,684 PM [test-shard-dispatcher-25] [INFO] Shard - member-1-shard-shard2-config52: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:14,684 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config52: Initializing shard [shard2] 01:49:14,684 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-shard2-config52 is initialized 01:49:14,684 PM [test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard2-config52: onLeadershipChange, isLeader: false, hasLeader : false 01:49:14,684 PM [test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-shard2-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-shard2-config52 is no longer the leader 01:49:14,684 PM [test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config52 , received role change from null to Follower 01:49:14,685 PM [test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-shard2-config52 (Follower): Received TimeoutNow - switching to Candidate 01:49:14,685 PM [test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-shard2-config52 (Candidate): Election: Candidate has following voting peers: [] 01:49:14,685 PM [test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-shard2-config52: Set currentTerm=1, votedFor=member-1-shard-shard2-config52 01:49:14,685 PM [test-shard-dispatcher-25] [INFO] Shard - member-1-shard-shard2-config52 (Candidate): Starting new election term 1 01:49:14,685 PM [test-shard-dispatcher-25] [INFO] Shard - member-1-shard-shard2-config52 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:49:14,685 PM [test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-1-shard-shard2-config52 (Follower): Closing 01:49:14,685 PM [test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard2-config52: onLeadershipChange, isLeader: false, hasLeader : false 01:49:14,685 PM [test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-shard2-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-shard2-config52 is no longer the leader 01:49:14,685 PM [test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config52 , received role change from Follower to Candidate 01:49:14,686 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config52 (Candidate): Received ElectionTimeout 01:49:14,686 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config52 (Leader): Election: Leader has following peers: [] 01:49:14,686 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config52: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 01:49:14,686 PM [test-shard-dispatcher-19] [INFO] Shard - member-1-shard-shard2-config52 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 01:49:14,686 PM [test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard2-config52: onLeadershipChange, isLeader: true, hasLeader : true 01:49:14,686 PM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-shard1-config52: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:14,686 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config52: Initializing shard [shard1] 01:49:14,686 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config52: became leader with frontend state for [] 01:49:14,686 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-shard1-config52 is initialized 01:49:14,686 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-shard1-config52 is initialized - notifying 1 OnShardInitialized callbacks 01:49:14,686 PM [test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config52 , received role change from Candidate to Leader 01:49:14,686 PM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard1-config52: onLeadershipChange, isLeader: false, hasLeader : false 01:49:14,686 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-shard1-config52 is no longer the leader 01:49:14,686 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config52: findLocalShard : shard2 01:49:14,686 PM [test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config52 , registered listener akka://test/user/$$s 01:49:14,686 PM [test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config52 , received role change from null to Follower 01:49:14,687 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config52 (Follower): Received TimeoutNow - switching to Candidate 01:49:14,687 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config52: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-shard2-config52, leaderId=member-1-shard-shard2-config52, leaderPayloadVersion=13] 01:49:14,687 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config52 (Candidate): Election: Candidate has following voting peers: [] 01:49:14,687 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config52: Set currentTerm=1, votedFor=member-1-shard-shard1-config52 01:49:14,687 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config52: Received role changed for member-1-shard-shard2-config52 from Candidate to Leader 01:49:14,687 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config52: onGetSnapshot 01:49:14,687 PM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-shard1-config52 (Candidate): Starting new election term 1 01:49:14,687 PM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-shard1-config52 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:49:14,687 PM [test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-1-shard-shard1-config52 (Follower): Closing 01:49:14,687 PM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard1-config52: onLeadershipChange, isLeader: false, hasLeader : false 01:49:14,687 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-shard1-config52 is no longer the leader 01:49:14,687 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Expecting 2 shard snapshot replies 01:49:14,687 PM [test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config52 , received role change from Follower to Candidate 01:49:14,687 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config52: onGetSnapshot 01:49:14,687 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config52: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 01:49:14,687 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config52: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 01:49:14,687 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Received GetSnapshotReply [id=member-1-shard-shard2-config52, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@11abb851, electionTerm=1, electionVotedFor=member-1-shard-shard2-config52, ServerConfigPayload=null]] 01:49:14,688 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config52 (Candidate): Received ElectionTimeout 01:49:14,688 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config52 (Leader): Election: Leader has following peers: [] 01:49:14,688 PM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-shard1-config52 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 01:49:14,688 PM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard1-config52: onLeadershipChange, isLeader: true, hasLeader : true 01:49:14,688 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config52: became leader with frontend state for [] 01:49:14,688 PM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config52 , received role change from Candidate to Leader 01:49:14,688 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config52: onGetSnapshot 01:49:14,688 PM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config52 , registered listener akka://test/user/$$s 01:49:14,688 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config52: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 01:49:14,688 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config52: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-shard1-config52, leaderId=member-1-shard-shard1-config52, leaderPayloadVersion=13] 01:49:14,688 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config52: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 01:49:14,688 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config52: Received role changed for member-1-shard-shard1-config52 from Candidate to Leader 01:49:14,688 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config52: All Shards are ready - data store config52 is ready 01:49:14,689 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Received GetSnapshotReply [id=member-1-shard-shard1-config52, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@6a8e655c, electionTerm=1, electionVotedFor=member-1-shard-shard1-config52, ServerConfigPayload=null]] 01:49:14,689 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: All shard snapshots received 01:49:14,700 PM [test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config52: onAddShardReplica: AddShardReplica[ShardName=astronauts] 01:49:14,701 PM [test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config52: Sending AddServer message to peer akka://test@127.0.0.1:2565/system/testProbe-631#1567592190 for shard member-1-shard-astronauts-config52 01:49:14,701 PM [test-shard-dispatcher-25] [INFO] Shard - Shard created : member-1-shard-astronauts-config52, persistent : true 01:49:14,701 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config52: Received AddServerReply{status=OK, leaderHint=} for shard astronauts from leader akka://test@127.0.0.1:2565/system/testProbe-631#1567592190 01:49:14,701 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config52: Leader shard successfully added the replica shard astronauts 01:49:14,702 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-astronauts-config52 01:49:14,702 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config52: persisting the shard list [shard2, astronauts, shard1] 01:49:14,702 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config52: findLocalShard : astronauts 01:49:14,702 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config52: Scheduling 600 ms timer to wait for shard ShardInformation [shardId=member-1-shard-astronauts-config52, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 01:49:14,702 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config52 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 01:49:14,702 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$s/member-1-shard-astronauts-config52/shard-member-1:snapshot-read#907690696] 01:49:14,702 PM [test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$s/member-1-shard-astronauts-config52/member-1-shard-astronauts-config52-notifier#1390876229 created and ready for shard:member-1-shard-astronauts-config52 01:49:14,703 PM [test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config52: Successfully deleted prior snapshots 01:49:14,703 PM [test-shard-dispatcher-25] [INFO] Shard - Starting recovery for member-1-shard-astronauts-config52 with journal batch size 1 01:49:14,705 PM [test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-astronauts-config52: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 01:49:14,705 PM [test-shard-dispatcher-25] [INFO] Shard - member-1-shard-astronauts-config52: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:14,705 PM [test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config52: Initializing shard [astronauts] 01:49:14,705 PM [test-akka.actor.default-dispatcher-23] [DEBUG] ShardInformation - Shard member-1-shard-astronauts-config52 is initialized 01:49:14,705 PM [test-akka.actor.default-dispatcher-23] [DEBUG] ShardInformation - Shard member-1-shard-astronauts-config52 is initialized - notifying 1 OnShardInitialized callbacks 01:49:14,705 PM [test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config52: onLeadershipChange, isLeader: false, hasLeader : false 01:49:14,705 PM [test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-astronauts-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-astronauts-config52 is no longer the leader 01:49:14,705 PM [test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config52 , received role change from null to Follower 01:49:14,705 PM [test-shard-dispatcher-25] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:49:14,705 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config52: onGetSnapshot 01:49:14,705 PM [test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-astronauts-config52: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 01:49:14,705 PM [test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-astronauts-config52: Re-initializing to Follower with previous leaderId null 01:49:14,705 PM [test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-1-shard-astronauts-config52 (Follower): Closing 01:49:14,705 PM [test-akka.actor.default-dispatcher-23] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Expecting 3 shard snapshot replies 01:49:14,705 PM [test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config52: onLeadershipChange, isLeader: false, hasLeader : false 01:49:14,705 PM [test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-astronauts-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-astronauts-config52 is no longer the leader 01:49:14,705 PM [test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-astronauts-config52 (Follower): Received TimeoutNow - switching to Candidate 01:49:14,706 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config52: onGetSnapshot 01:49:14,706 PM [test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-astronauts-config52 (Candidate): Election: Candidate has following voting peers: [] 01:49:14,706 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config52: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 01:49:14,706 PM [test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-astronauts-config52: Set currentTerm=1, votedFor=member-1-shard-astronauts-config52 01:49:14,706 PM [test-shard-dispatcher-25] [INFO] Shard - member-1-shard-astronauts-config52 (Candidate): Starting new election term 1 01:49:14,706 PM [test-shard-dispatcher-25] [INFO] Shard - member-1-shard-astronauts-config52 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:49:14,706 PM [test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-1-shard-astronauts-config52 (Follower): Closing 01:49:14,706 PM [test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config52: onLeadershipChange, isLeader: false, hasLeader : false 01:49:14,706 PM [test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-astronauts-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-astronauts-config52 is no longer the leader 01:49:14,706 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config52: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 01:49:14,706 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config52: onGetSnapshot 01:49:14,706 PM [test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config52 , received role change from Follower to Candidate 01:49:14,706 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config52: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 01:49:14,706 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config52: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 01:49:14,706 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Received GetSnapshotReply [id=member-1-shard-shard2-config52, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@637c7c9a, electionTerm=1, electionVotedFor=member-1-shard-shard2-config52, ServerConfigPayload=null]] 01:49:14,706 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Received GetSnapshotReply [id=member-1-shard-shard1-config52, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@33b9499b, electionTerm=1, electionVotedFor=member-1-shard-shard1-config52, ServerConfigPayload=null]] 01:49:14,707 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config52 (Candidate): Received ElectionTimeout 01:49:14,707 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config52 (Leader): Election: Leader has following peers: [] 01:49:14,707 PM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-astronauts-config52 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 01:49:14,707 PM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config52: onLeadershipChange, isLeader: true, hasLeader : true 01:49:14,707 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config52: became leader with frontend state for [] 01:49:14,707 PM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config52 , received role change from Candidate to Leader 01:49:14,707 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config52: onGetSnapshot 01:49:14,707 PM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config52 , registered listener akka://test/user/$$s 01:49:14,708 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config52: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 01:49:14,708 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config52: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-astronauts-config52, leaderId=member-1-shard-astronauts-config52, leaderPayloadVersion=13] 01:49:14,708 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config52: Received role changed for member-1-shard-astronauts-config52 from Candidate to Leader 01:49:14,708 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config52: All Shards are ready - data store config52 is ready 01:49:14,708 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config52: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 01:49:14,708 PM [test-akka.actor.default-dispatcher-23] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Received GetSnapshotReply [id=member-1-shard-astronauts-config52, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@306f14f8, electionTerm=1, electionVotedFor=member-1-shard-astronauts-config52, ServerConfigPayload=null]] 01:49:14,708 PM [test-akka.actor.default-dispatcher-23] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: All shard snapshots received 01:49:14,708 PM [main] [INFO] ShardManagerTest - testGetSnapshot ending 01:49:14,810 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$s] 01:49:14,821 PM [test-shard-dispatcher-19] [INFO] Shard - Stopping Shard member-1-shard-astronauts-config52 01:49:14,821 PM [test-shard-dispatcher-19] [DEBUG] MessageSlicer - member-1-shard-astronauts-config52 (Leader)_slicer-id-240: Closing 01:49:14,821 PM [test-shard-dispatcher-18] [INFO] Shard - Stopping Shard member-1-shard-shard2-config52 01:49:14,821 PM [test-shard-dispatcher-24] [INFO] Shard - Stopping Shard member-1-shard-shard1-config52 01:49:14,821 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config52: Aborting 0 pending queued transactions 01:49:14,821 PM [test-shard-dispatcher-18] [DEBUG] MessageSlicer - member-1-shard-shard2-config52 (Leader)_slicer-id-237: Closing 01:49:14,821 PM [test-shard-dispatcher-24] [DEBUG] MessageSlicer - member-1-shard-shard1-config52 (Leader)_slicer-id-238: Closing 01:49:14,821 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config52: Aborting 0 pending queued transactions 01:49:14,821 PM [test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-shard1-config52: Aborting 0 pending queued transactions 01:49:14,821 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config52 01:49:14,823 PM [main] [INFO] ShardManagerTest - testWhenMultipleShardsPresentSyncStatusMustBeTrueForAllShards starting 01:49:14,827 PM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config53 01:49:14,828 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config53 01:49:14,828 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config53: Creating local shard: member-1-shard-default-config53 01:49:14,828 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config53: Creating local shard: member-1-shard-astronauts-config53 01:49:14,829 PM [main] [INFO] ShardManager - shard-manager-config53: Received role changed for member-1-shard-default-config53 from Follower to Leader 01:49:14,829 PM [main] [INFO] ShardManager - shard-manager-config53: Received role changed for member-1-shard-astronauts-config53 from Follower to Leader 01:49:14,829 PM [main] [INFO] ShardManager - shard-manager-config53: Received role changed for member-1-shard-astronauts-config53 from Leader to Follower 01:49:14,829 PM [main] [INFO] ShardManager - shard-manager-config53 Received follower initial sync status for member-1-shard-astronauts-config53 status sync done true 01:49:14,829 PM [main] [INFO] ShardManagerTest - testWhenMultipleShardsPresentSyncStatusMustBeTrueForAllShards ending 01:49:14,930 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$t] 01:49:14,940 PM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config53 01:49:14,942 PM [main] [INFO] ShardManagerTest - testAddShardReplicaWithFindPrimaryTimeout starting 01:49:14,944 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config54 01:49:14,946 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config54 01:49:14,955 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:49:14,955 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config54: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://non-existent@127.0.0.1:5 01:49:14,955 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config54: All Shards are ready - data store config54 is ready 01:49:14,955 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config54: onAddShardReplica: AddShardReplica[ShardName=astronauts] 01:49:14,955 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config54: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=astronauts, waitUntilReady=true] 01:49:14,955 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config54: findPrimary for astronauts forwarding to remote ShardManager akka://non-existent@127.0.0.1:5/user/shardmanager-config54, visitedAddresses: [akka://test@127.0.0.1:2550/user/shardmanager-config54] 01:49:14,972 PM [test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://non-existent@127.0.0.1:5], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:5,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 01:49:14,972 PM [test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://non-existent@127.0.0.1:5], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:5,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 01:49:15,174 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config54: Received failure from FindPrimary for shard astronauts akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://test/user/shardmanager-config54#-1692056233]] after [200 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.FindPrimary]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:49:15,174 PM [main] [INFO] ShardManagerTest - testAddShardReplicaWithFindPrimaryTimeout ending 01:49:15,276 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/shardmanager-config54#-1692056233] 01:49:15,286 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config54 01:49:15,288 PM [main] [INFO] ShardManagerTest - testOnCreateShard starting 01:49:15,298 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config55 01:49:15,300 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config55 01:49:15,300 PM [test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 01:49:15,308 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config55: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=foo-ns, moduleName=foo-module, shardName=foo, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-5}, MemberName{name=member-6}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@3c5334d8] 01:49:15,308 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config55 doCreateShard: shardId: member-1-shard-foo-config55, memberNames: [MemberName{name=member-1}, MemberName{name=member-5}, MemberName{name=member-6}], peerAddresses: {member-6-shard-foo-config55=null, member-5-shard-foo-config55=null}, isActiveMember: true 01:49:15,309 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config55: findLocalShard : foo 01:49:15,309 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config55: Scheduling 300000 ms timer to wait for shard ShardInformation [shardId=member-1-shard-foo-config55, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 01:49:15,310 PM [test-shard-dispatcher-24] [INFO] Shard - Shard created : member-1-shard-foo-config55, persistent : false 01:49:15,310 PM [test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$C/member-1-shard-foo-config55/member-1-shard-foo-config55-notifier#-1047801061 created and ready for shard:member-1-shard-foo-config55 01:49:15,310 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$C/member-1-shard-foo-config55/shard-member-1:snapshot-read#-482631243] 01:49:15,311 PM [test-shard-dispatcher-24] [INFO] Shard - Starting recovery for member-1-shard-foo-config55 with journal batch size 1 01:49:15,314 PM [test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-foo-config55: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 01:49:15,314 PM [test-shard-dispatcher-24] [INFO] Shard - member-1-shard-foo-config55: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:15,314 PM [test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-foo-config55: onLeadershipChange, isLeader: false, hasLeader : false 01:49:15,314 PM [test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-foo-config55: onStateChanged: Closing all transaction chains because shard member-1-shard-foo-config55 is no longer the leader 01:49:15,315 PM [test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config55 , received role change from null to Follower 01:49:15,315 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config55: Initializing shard [foo] 01:49:15,315 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-foo-config55 is initialized 01:49:15,315 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-foo-config55 is initialized - notifying 1 OnShardInitialized callbacks 01:49:15,315 PM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config55 , registered listener akka://test/user/$C 01:49:15,315 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config55: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=foo-ns, moduleName=foo-module, shardName=foo, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-5}, MemberName{name=member-6}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@3c5334d8] 01:49:15,315 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config55: Shard foo already exists 01:49:15,315 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config55: Received role changed for member-1-shard-foo-config55 from null to Follower 01:49:15,315 PM [main] [INFO] ShardManagerTest - testOnCreateShard ending 01:49:15,417 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$C#259014606] 01:49:15,428 PM [test-shard-dispatcher-24] [INFO] Shard - Stopping Shard member-1-shard-foo-config55 01:49:15,428 PM [test-shard-dispatcher-24] [DEBUG] MessageAssembler - member-1-shard-foo-config55 (Follower): Closing 01:49:15,428 PM [test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-foo-config55: Aborting 0 pending queued transactions 01:49:15,428 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Stopping ShardManager shard-manager-config55 01:49:15,429 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config1 01:49:15,429 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Stopping ShardManager shard-manager-config35 01:49:15,429 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config43 01:49:15,429 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Stopping ShardManager shard-manager-config11 01:49:15,429 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config21 01:49:15,429 PM [test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Stopping ShardManager shard-manager-config46 01:49:15,430 PM [test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:15,430 PM [test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:15,441 PM [test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:15.448] [test-akka.actor.internal-dispatcher-5] [Cluster(akka://test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config