04:01:44,533 PM [test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 04:01:44,544 PM [test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://test@127.0.0.1:2565] with UID [4539565305954159873] 04:01:44,545 PM [test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://test@127.0.0.1:2565] - Starting up, Akka version [2.6.21] ... 04:01:44,550 PM [test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://test@127.0.0.1:2565] - Registered cluster JMX MBean [akka:type=Cluster] 04:01:44,550 PM [test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://test@127.0.0.1:2565] - Started up successfully 04:01:44,552 PM [test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://test@127.0.0.1:2565] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 04:01:44,552 PM [test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://test@127.0.0.1:2565] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 04:01:44,720 PM [test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$a#665117643] to Actor[akka://test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 04:01:44,723 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config1 04:01:44,725 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shard-manager-config1 04:01:44,725 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config1: Creating local shard: member-1-shard-default-config1 04:01:44,729 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:01:44,729 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config1 04:01:44,729 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config1: Initializing shard [default] 04:01:44,729 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-default-config1 is initialized 04:01:44,729 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config1: Received role changed for member-1-shard-default-config1 from null to Follower 04:01:44,729 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config1: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=default, meberVotingStatusMap={member-2=true}] 04:01:44,732 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config1: findLocalShard : default 04:01:44,735 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-default-config1=true}, serversVisited=[]] to local shard akka://test/user/member-1-shard-default-config1 04:01:44,737 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config1: Received ServerChangeReply{status=NO_LEADER} from local shard akka://test/user/member-1-shard-default-config1 04:01:44,737 PM [test-akka.actor.default-dispatcher-11] [WARN] ShardManager - shard-manager-config1: ChangeServersVotingStatus failed for shard default with status NO_LEADER 04:01:44,886 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-default-config1#749632300] 04:01:44,901 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForShardLeader starting 04:01:44,906 PM [test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$b#431102692] to Actor[akka://test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 04:01:44,906 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config2 04:01:44,908 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config2 04:01:44,908 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config2: Creating local shard: member-1-shard-default-config2 04:01:44,917 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:01:44,917 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config2 04:01:44,917 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 04:01:44,917 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config2: Scheduling 10000 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config2, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 04:01:45,67 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config2: Initializing shard [default] 04:01:45,67 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-default-config2 is initialized 04:01:45,67 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-default-config2 is initialized - notifying 1 OnShardInitialized callbacks 04:01:45,217 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config2: Received role changed for member-1-shard-default-config2 from Candidate to Leader 04:01:45,217 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-default-config2 is initialized - notifying 1 OnShardInitialized callbacks 04:01:45,368 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config2, leaderId=member-1-shard-default-config2, leaderPayloadVersion=13] 04:01:45,368 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-default-config2 is ready - notifying 1 OnShardInitialized callbacks 04:01:45,369 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config2: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://test@127.0.0.1:2565/user/member-1-shard-default-config#2044673077, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$mziea4nU@323384a5] 04:01:45,369 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config2: All Shards are ready - data store config2 is ready 04:01:45,570 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForShardLeader ending 04:01:45,672 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$b#431102692] 04:01:45,683 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config2 04:01:45,692 PM [test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://test/user/$$b] to Actor[akka://test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 04:01:45,692 PM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config3 04:01:45,696 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config3 04:01:45,696 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config3: Creating local shard: member-1-shard-default-config3 04:01:45,696 PM [main] [INFO] ShardManager - shard-manager-config3: Received role changed for member-1-shard-default-config3 from Follower to Candidate 04:01:45,697 PM [main] [INFO] ShardManager - shard-manager-config3 Received follower initial sync status for member-1-shard-default-config3 status sync done true 04:01:45,798 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$b] 04:01:45,810 PM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config3 04:01:45,826 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 04:01:45,839 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-6057419527919887633] 04:01:45,839 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 04:01:45,849 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 04:01:45,850 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 04:01:45,858 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 04:01:45,858 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 04:01:45,880 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Starting ShardManager shard-manager-config4 04:01:45,880 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 04:01:45,880 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config4] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 04:01:45,885 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config4 04:01:45,885 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config4: Creating local shard: member-1-shard-default-config4 04:01:45,886 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config4: Creating local shard: member-1-shard-astronauts-config4 04:01:45,887 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 04:01:45,891 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 04:01:45,895 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardManager - shard-manager-config4: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 04:01:45,895 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-default-config4 04:01:45,895 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-astronauts-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-astronauts-config4 04:01:45,903 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 04:01:45,923 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [1607010535594827547] 04:01:45,923 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 04:01:45,926 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 04:01:45,926 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 04:01:45,930 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 04:01:45,930 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 04:01:45,945 PM [main] [ERROR] ShardManagerTest - Mock Shard Leader Actor : TestActor[akka://cluster-test/user/member-2-shard-default-config4] 04:01:45,956 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config4 04:01:45,956 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config4] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 04:01:45,958 PM [main] [ERROR] ShardManagerTest - Forwarding actor : Actor[akka://cluster-test/user/shardmanager-config4/member-2-shard-default-config4#1596120644] 04:01:45,971 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:01:45,971 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config4 04:01:45,971 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-default-config4 04:01:45,971 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-default-config4 to akka://cluster-test/user/member-1-shard-default-config 04:01:45,971 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config4: updated peer member-1-shard-default-config4 on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-default-config4 on shard member-1-shard-default-config4 whose actor address is Actor[akka://cluster-test/user/member-1-shard-default-config#566047071] 04:01:45,971 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-astronauts-config4 04:01:45,971 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-astronauts-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-astronauts-config4 04:01:45,971 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-astronauts-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-astronauts-config4 to akka://cluster-test/user/member-1-shard-default-config 04:01:45,971 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config4: updated peer member-1-shard-astronauts-config4 on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-astronauts-config4 on shard member-1-shard-astronauts-config4 whose actor address is Actor[akka://cluster-test/user/member-1-shard-default-config#566047071] 04:01:45,974 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config4: Initializing shard [default] 04:01:45,974 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-default-config4 is initialized 04:01:45,974 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config4: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config4, leaderId=member-2-shard-default-config4, leaderPayloadVersion=12] 04:01:45,974 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config4: Received role changed for member-1-shard-default-config4 from Candidate to Follower 04:01:45,982 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config4 04:01:45,982 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config4: Creating local shard: member-2-shard-default-config4 04:01:45,983 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:01:45,983 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-2-shard-default-config4 04:01:45,984 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config4: Initializing shard [default] 04:01:45,984 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Shard member-2-shard-default-config4 is initialized 04:01:45,984 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config4: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-config4, leaderId=member-2-shard-default-config4, leaderPayloadVersion=12] 04:01:45,984 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config4: Received role changed for member-2-shard-default-config4 from Candidate to Leader 04:01:45,984 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config4: All Shards are ready - data store config4 is ready 04:01:46,309 PM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@61de178c aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-3-1, state=OPEN} 04:01:46,310 PM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-3-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1546449688]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1524/0x00000008014a7000@fced187 04:01:46,310 PM [Cleaner-0] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2bc21806 transmitting entry ConnectionEntry{request=AbortLocalTransactionRequest{target=member-1-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1546449688]}, enqueuedTicks=632277449141} 04:01:46,310 PM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@74447448 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-3-2, state=OPEN} 04:01:46,310 PM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-3-2, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-3-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1546449688]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1524/0x00000008014a7000@2fa32fa3 04:01:46,310 PM [Cleaner-0] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@1f162aea transmitting entry ConnectionEntry{request=AbortLocalTransactionRequest{target=member-1-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-3-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1546449688]}, enqueuedTicks=632277554893} 04:01:46,310 PM [Cleaner-0] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-1-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-3-0} 04:01:46,353 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 04:01:46,425 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 04:01:46,425 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config4: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 04:01:46,425 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-default-config4 04:01:46,425 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-default-config4 to akka://cluster-test/user/member-2-shard-default-config4 04:01:46,425 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config4: All Shards are ready - data store config4 is ready 04:01:46,877 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 04:01:46,883 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config4: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 04:01:46,883 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config4 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-default-config4 04:01:46,883 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config4 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-default-config4 to akka://cluster-test/user/member-1-shard-default-config 04:01:46,883 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-astronauts-config4 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-astronauts-config4 04:01:46,883 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-astronauts-config4 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-astronauts-config4 to akka://cluster-test/user/member-1-shard-default-config 04:01:46,884 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config4: onRemoveShardReplica: RemoveShardReplica [shardName=default, memberName=MemberName{name=member-1}] 04:01:46,887 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config4: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 04:01:46,887 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config4: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-default-config4, primaryVersion=12] 04:01:46,888 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config4: Sending RemoveServer message to peer akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-default-config4 for shard member-1-shard-default-config4 04:01:46,918 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config4: Received RemoveServerReply{status=OK, leaderHint=member-2-shard-default-config4} for shard default 04:01:46,918 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config4: Leader shard successfully removed the replica shard default 04:01:46,953 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config4: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 04:01:46,953 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config4 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-default-config4 04:01:46,953 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config4 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-default-config4 to akka://cluster-test/user/member-2-shard-default-config4 04:01:46,953 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config4: All Shards are ready - data store config4 is ready 04:01:47,41 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Stopping ShardManager shard-manager-config4 04:01:47,42 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 04:01:47,42 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 04:01:47,70 PM [cluster-test-akka.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 04:01:47,70 PM [cluster-test-akka.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 04:01:47,76 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 04:01:47,83 PM [cluster-test-akka.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection closed with error: Connection reset 04:01:47,83 PM [cluster-test-akka.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection closed with error: Connection reset 04:01:47,116 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config4 04:01:47,118 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 04:01:47,118 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 04:01:48,151 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [01/16/2024 16:01:48.155] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 04:01:48,180 PM [test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$c#-439630618] to Actor[akka://test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 04:01:48,181 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config5 04:01:48,183 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shard-manager-config5 04:01:48,183 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config5: Creating local shard: member-1-shard-default-config5 04:01:48,192 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:01:48,192 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config5 04:01:48,192 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config5: Initializing shard [default] 04:01:48,192 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-default-config5 is initialized 04:01:48,344 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$c#-439630618] 04:01:48,355 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config5 04:01:48,365 PM [test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://test/user/$$c] to Actor[akka://test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 04:01:48,365 PM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config6 04:01:48,367 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shard-manager-config6 04:01:48,367 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config6: Creating local shard: member-1-shard-default-config6 04:01:48,368 PM [main] [INFO] ShardManager - shard-manager-config6: Received role changed for member-1-shard-default-config6 from null to Follower 04:01:48,378 PM [main] [INFO] ShardManager - shard-manager-config6: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://test 04:01:48,378 PM [main] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config6 with address akka://test/user/shardmanager-config6/member-2-shard-default-config6 04:01:48,379 PM [main] [INFO] ShardManager - shard-manager-config6: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config6, leaderId=member-2-shard-default-config6, leaderPayloadVersion=13] 04:01:48,379 PM [main] [INFO] ShardManager - shard-manager-config6: All Shards are ready - data store config6 is ready 04:01:48,481 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$c] 04:01:48,492 PM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config6 04:01:48,502 PM [test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$d#-2066118024] to Actor[akka://test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 04:01:48,502 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config7 04:01:48,504 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config7 04:01:48,513 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config7: onRemoveShardReplica: RemoveShardReplica [shardName=model-inventory, memberName=MemberName{name=member-1}] 04:01:48,513 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config7: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=model-inventory, waitUntilReady=true] 04:01:48,513 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config7: No shard found for model-inventory 04:01:48,514 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config7: Failed to find leader for shard model-inventory: received response: {} org.opendaylight.controller.cluster.datastore.exceptions.PrimaryNotFoundException: No primary shard found for model-inventory. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.findPrimary(ShardManager.java:1051) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:207) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManagerTest$TestShardManager.handleCommand(ShardManagerTest.java:2198) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 04:01:48,616 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$d#-2066118024] 04:01:48,627 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config7 04:01:48,631 PM [main] [INFO] ShardManagerTest - testAddShardReplicaWithPreExistingLocalReplicaLeader starting 04:01:48,634 PM [test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$e#1716878606] to Actor[akka://test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 04:01:48,635 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config8 04:01:48,638 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shard-manager-config8 04:01:48,638 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config8: Creating local shard: member-1-shard-default-config8 04:01:48,647 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:01:48,647 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config8 04:01:48,647 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config8: Initializing shard [default] 04:01:48,647 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config8 is initialized 04:01:48,647 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config8: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config8, leaderId=member-1-shard-default-config8, leaderPayloadVersion=13] 04:01:48,648 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config8: Received role changed for member-1-shard-default-config8 from Candidate to Leader 04:01:48,648 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config8: All Shards are ready - data store config8 is ready 04:01:48,648 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config8: onAddShardReplica: AddShardReplica[ShardName=default] 04:01:48,649 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config8: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 04:01:48,650 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config8: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://test@127.0.0.1:2565/user/member-1-shard-default-config#-488315406, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$mziea4nU@5dc7d016] 04:01:48,650 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config8: Local shard default already exists 04:01:48,651 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config8: findLocalShard : default 04:01:48,651 PM [main] [INFO] ShardManagerTest - testAddShardReplicaWithPreExistingLocalReplicaLeader ending 04:01:48,753 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$e#1716878606] 04:01:48,765 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config8 04:01:48,768 PM [main] [INFO] ShardManagerTest - testOnReceiveFindLocalShardWaitForShardInitialized starting 04:01:48,771 PM [test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$f#2093668683] to Actor[akka://test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 04:01:48,772 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config9 04:01:48,779 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config9 04:01:48,779 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config9: Creating local shard: member-1-shard-default-config9 04:01:48,783 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:01:48,783 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config9 04:01:48,783 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config9: findLocalShard : default 04:01:48,783 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config9: Scheduling 600 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config9, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 04:01:48,783 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config9: Initializing shard [default] 04:01:48,783 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config9 is initialized 04:01:48,783 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config9 is initialized - notifying 1 OnShardInitialized callbacks 04:01:48,786 PM [main] [INFO] ShardManagerTest - testOnReceiveFindLocalShardWaitForShardInitialized starting 04:01:48,888 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$f#2093668683] 04:01:48,899 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config9 04:01:48,901 PM [main] [INFO] ShardManagerTest - testShardAvailabilityChangeOnMemberUnreachableAndLeadershipChange starting 04:01:48,910 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 04:01:48,922 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [5422972321572784919] 04:01:48,923 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 04:01:48,925 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 04:01:48,925 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 04:01:48,929 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 04:01:48,929 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 04:01:48,941 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 04:01:48,941 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 04:01:48,941 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 04:01:48,942 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config10] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 04:01:48,945 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config10 04:01:48,968 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 04:01:48,975 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shard-manager-config10 04:01:48,975 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config10: Creating local shard: member-1-shard-default-config10 04:01:48,976 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config10: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 04:01:48,976 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config10 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config10/member-1-shard-default-config10 04:01:49,12 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-4833068470708963510] 04:01:49,12 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 04:01:49,20 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 04:01:49,20 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 04:01:49,20 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 04:01:49,20 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 04:01:49,47 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:01:49,47 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config10 04:01:49,47 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config10 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config10/member-1-shard-default-config10 04:01:49,47 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config10 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config10/member-1-shard-default-config10 to akka://cluster-test/user/member-1-shard-default-config 04:01:49,47 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config10: updated peer member-1-shard-default-config10 on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config10/member-1-shard-default-config10 on shard member-1-shard-default-config10 whose actor address is Actor[akka://cluster-test/user/member-1-shard-default-config#508160205] 04:01:49,47 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config10: Initializing shard [default] 04:01:49,47 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-default-config10 is initialized 04:01:49,47 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config10: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config10, leaderId=member-2-shard-default-config10, leaderPayloadVersion=13] 04:01:49,47 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config10: Received role changed for member-1-shard-default-config10 from Candidate to Follower 04:01:49,49 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config10] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 04:01:49,54 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config10 04:01:49,69 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-config10 04:01:49,69 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config10: Creating local shard: member-2-shard-default-config10 04:01:49,70 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:01:49,70 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-2-shard-default-config10 04:01:49,70 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config10: Initializing shard [default] 04:01:49,70 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardInformation - Shard member-2-shard-default-config10 is initialized 04:01:49,70 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config10: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-config10, leaderId=member-2-shard-default-config10, leaderPayloadVersion=13] 04:01:49,70 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config10: Received role changed for member-2-shard-default-config10 from Candidate to Leader 04:01:49,70 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config10: All Shards are ready - data store config10 is ready 04:01:49,144 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 04:01:49,150 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 04:01:49,157 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config10: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 04:01:49,157 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config10 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config10/member-1-shard-default-config10 04:01:49,157 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config10 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config10/member-1-shard-default-config10 to akka://cluster-test/user/member-2-shard-default-config 04:01:49,157 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config10: All Shards are ready - data store config10 is ready 04:01:49,949 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 04:01:49,951 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config10: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 04:01:49,951 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config10 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config10/member-2-shard-default-config10 04:01:49,951 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config10 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config10/member-2-shard-default-config10 to akka://cluster-test/user/member-1-shard-default-config 04:01:49,951 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config10: All Shards are ready - data store config10 is ready 04:01:49,951 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config10: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 04:01:49,951 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config10: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config10/member-2-shard-default-config10, primaryVersion=13] 04:01:49,953 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 04:01:49,953 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Marking Leader member-2-shard-default-config10 as unavailable. 04:01:49,953 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config10: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 04:01:49,953 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config10: Scheduling 1200 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config10, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=member-2-shard-default-config10, activeMember=true] 04:01:50,48 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config10: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 04:01:50,49 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config10 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config10/member-2-shard-default-config10 04:01:50,49 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config10 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config10/member-2-shard-default-config10 to akka://cluster-test/user/member-2-shard-default-config 04:01:50,49 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config10: All Shards are ready - data store config10 is ready 04:01:51,167 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config10: Received ShardNotInitializedTimeout message for shard default 04:01:51,167 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config10: Returning NoShardLeaderException for shard default 04:01:51,168 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config10: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config10, leaderId=member-1-shard-default-config10, leaderPayloadVersion=13] 04:01:51,169 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config10: All Shards are ready - data store config10 is ready 04:01:51,169 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config10: Received role changed for member-1-shard-default-config10 from Follower to Leader 04:01:51,169 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config10: All Shards are ready - data store config10 is ready 04:01:51,169 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config10: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 04:01:51,169 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config10: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/member-1-shard-default-config#508160205, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$mziea4nU@68628047] 04:01:51,169 PM [main] [INFO] ShardManagerTest - testShardAvailabilityChangeOnMemberUnreachableAndLeadershipChange ending 04:01:51,272 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config10 04:01:51,273 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 04:01:51,273 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 04:01:51,284 PM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 04:01:51,284 PM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 04:01:51,287 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 04:01:51,321 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - Stopping ShardManager shard-manager-config10 04:01:51,324 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 04:01:51,324 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 04:01:52,348 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [01/16/2024 16:01:52.350] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 04:01:52,372 PM [test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://test/user/shardmanager-config11] to Actor[akka://test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 04:01:52,372 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config11 04:01:52,376 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config11 04:01:52,397 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:01:52,397 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config11: onAddShardReplica: AddShardReplica[ShardName=astronauts] 04:01:52,405 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config11: Sending AddServer message to peer akka://test@127.0.0.1:2565/system/testProbe-27#194814992 for shard member-1-shard-astronauts-config11 04:01:52,416 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config11: onRemoveShardReplica: RemoveShardReplica [shardName=astronauts, memberName=MemberName{name=member-2}] 04:01:52,416 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config11: A shard replica operation for astronauts is already in progress 04:01:52,521 PM [main] [INFO] ShardManagerTest - testPerShardDatastoreContext starting 04:01:52,529 PM [test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$g#1148090710] to Actor[akka://test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 04:01:52,529 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config12 04:01:52,531 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config12 04:01:52,531 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config12: Creating local shard: member-1-shard-default-config12 04:01:52,531 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config12: Creating local shard: member-1-shard-topology-config12 04:01:52,540 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:01:52,540 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config12 04:01:52,540 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-topology-config12 04:01:52,542 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-default-config12 04:01:52,542 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-topology-config12 04:01:52,592 PM [main] [INFO] ShardManagerTest - testPerShardDatastoreContext ending 04:01:52,694 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/default1#-2058613617] 04:01:52,705 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/topology2#-863971526] 04:01:52,705 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$g#1148090710] 04:01:52,706 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config12 04:01:52,708 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForLocalLeaderShard starting 04:01:52,715 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config13 04:01:52,716 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config13 04:01:52,717 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config13: Creating local shard: member-1-shard-default-config13 04:01:52,726 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:01:52,726 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config13 04:01:52,726 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config13: Initializing shard [default] 04:01:52,726 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Shard member-1-shard-default-config13 is initialized 04:01:52,726 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config13: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config13, leaderId=member-1-shard-default-config13, leaderPayloadVersion=13] 04:01:52,726 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config13: Received role changed for member-1-shard-default-config13 from Candidate to Leader 04:01:52,726 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config13: All Shards are ready - data store config13 is ready 04:01:52,726 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config13: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 04:01:52,726 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config13: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://test@127.0.0.1:2565/user/member-1-shard-default-config#1959223015, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$mziea4nU@12f22145] 04:01:52,727 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForLocalLeaderShard ending 04:01:52,828 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$h#-318117609] 04:01:52,839 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config13 04:01:52,842 PM [main] [INFO] ShardManagerTest - testShardAvailabilityChangeOnMemberWithNameContainedInLeaderIdUnreachable starting 04:01:52,852 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 04:01:52,864 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2562] with UID [2611227126788561036] 04:01:52,864 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Starting up, Akka version [2.6.21] ... 04:01:52,867 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Registered cluster JMX MBean [akka:type=Cluster] 04:01:52,867 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Started up successfully 04:01:52,874 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 04:01:52,874 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 04:01:52,886 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Node [akka://cluster-test@127.0.0.1:2562] is JOINING itself (with roles [member-256, dc-default], version [0.0.0]) and forming new cluster 04:01:52,886 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - is the new leader among reachable nodes (more leaders may exist) 04:01:52,886 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Leader is moving node [akka://cluster-test@127.0.0.1:2562] to [Up] 04:01:52,886 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config14] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 04:01:52,893 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config14 04:01:52,898 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 04:01:52,909 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config14 04:01:52,909 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config14: Creating local shard: member-256-shard-default-config14 04:01:52,910 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config14: Received MemberUp: memberName: MemberName{name=member-256}, address: akka://cluster-test@127.0.0.1:2562 04:01:52,910 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-256-shard-default-config14 with address akka://cluster-test@127.0.0.1:2562/user/shardmanager-config14/member-256-shard-default-config14 04:01:52,924 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-8213873318920934930] 04:01:52,924 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 04:01:52,928 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 04:01:52,928 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 04:01:52,930 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 04:01:52,930 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 04:01:52,948 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config14] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 04:01:52,949 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Starting ShardManager shard-manager-config14 04:01:52,951 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shard-manager-config14 04:01:52,952 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config14: Creating local shard: member-2-shard-default-config14 04:01:52,953 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:01:52,953 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:01:52,953 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-default-config14 04:01:52,953 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-256-shard-default-config14 04:01:52,953 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-256-shard-default-config14 with address akka://cluster-test@127.0.0.1:2562/user/shardmanager-config14/member-256-shard-default-config14 04:01:52,953 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-256-shard-default-config14 with address akka://cluster-test@127.0.0.1:2562/user/shardmanager-config14/member-256-shard-default-config14 to akka://cluster-test/user/member-256-shard-default-config 04:01:52,953 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config14: Initializing shard [default] 04:01:52,953 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-default-config14 is initialized 04:01:52,953 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config14: updated peer member-256-shard-default-config14 on member MemberName{name=member-256} with address akka://cluster-test@127.0.0.1:2562/user/shardmanager-config14/member-256-shard-default-config14 on shard member-256-shard-default-config14 whose actor address is Actor[akka://cluster-test/user/member-256-shard-default-config#-541008149] 04:01:52,953 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config14: Initializing shard [default] 04:01:52,953 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Shard member-256-shard-default-config14 is initialized 04:01:52,954 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config14: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-config14, leaderId=member-256-shard-default-config14, leaderPayloadVersion=13] 04:01:52,954 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config14: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-256-shard-default-config14, leaderId=member-256-shard-default-config14, leaderPayloadVersion=13] 04:01:52,954 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config14: Received role changed for member-2-shard-default-config14 from Candidate to Follower 04:01:52,954 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config14: Received role changed for member-256-shard-default-config14 from Candidate to Leader 04:01:52,954 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config14: All Shards are ready - data store config14 is ready 04:01:53,16 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 04:01:53,21 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2562] 04:01:53,21 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config14: Received MemberUp: memberName: MemberName{name=member-256}, address: akka://cluster-test@127.0.0.1:2562 04:01:53,21 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-256-shard-default-config14 with address akka://cluster-test@127.0.0.1:2562/user/shardmanager-config14/member-256-shard-default-config14 04:01:53,21 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-256-shard-default-config14 with address akka://cluster-test@127.0.0.1:2562/user/shardmanager-config14/member-256-shard-default-config14 to akka://cluster-test/user/member-2-shard-default-config 04:01:53,21 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config14: All Shards are ready - data store config14 is ready 04:01:53,890 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 04:01:53,890 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config14: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 04:01:53,890 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config14 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config14/member-2-shard-default-config14 04:01:53,890 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config14 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config14/member-2-shard-default-config14 to akka://cluster-test/user/member-256-shard-default-config 04:01:53,890 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config14: All Shards are ready - data store config14 is ready 04:01:53,891 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config14: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 04:01:53,891 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config14: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2562/user/member-256-shard-default-config#-541008149, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$mziea4nU@64fbdf41] 04:01:53,892 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 04:01:53,893 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config14: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 04:01:53,893 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config14: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2562/user/member-256-shard-default-config#-541008149, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$mziea4nU@64fbdf41] 04:01:53,895 PM [main] [INFO] ShardManagerTest - testShardAvailabilityChangeOnMemberWithNameContainedInLeaderIdUnreachable ending 04:01:53,956 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config14: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 04:01:53,956 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config14 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config14/member-2-shard-default-config14 04:01:53,956 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config14 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config14/member-2-shard-default-config14 to akka://cluster-test/user/member-2-shard-default-config 04:01:53,956 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config14: All Shards are ready - data store config14 is ready 04:01:53,998 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config14 04:01:54,2 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 04:01:54,2 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 04:01:54,12 PM [cluster-test-akka.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 04:01:54,12 PM [cluster-test-akka.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 04:01:54,13 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 04:01:54,30 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Stopping ShardManager shard-manager-config14 04:01:54,32 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 04:01:54,32 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 04:01:54,964 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - is the new leader among reachable nodes (more leaders may exist) 04:01:55,46 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [01/16/2024 16:01:55.049] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 04:01:55,69 PM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config15 04:01:55,71 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shard-manager-config15 04:01:55,71 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config15: Creating local shard: member-1-shard-default-config15 04:01:55,71 PM [main] [INFO] ShardManager - shard-manager-config15: Received role changed for member-1-shard-default-config15 from Follower to Leader 04:01:55,173 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$d] 04:01:55,184 PM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config15 04:01:55,190 PM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config16 04:01:55,192 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config16 04:01:55,192 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config16: Creating local shard: member-1-shard-default-config16 04:01:55,193 PM [main] [INFO] ShardManager - shard-manager-config16: Received role changed for member-1-shard-default-config16 from Candidate to Leader 04:01:55,193 PM [main] [INFO] ShardManager - shard-manager-config16: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config16, leaderId=member-1-shard-default-config16, leaderPayloadVersion=13] 04:01:55,193 PM [main] [INFO] ShardManager - shard-manager-config16: All Shards are ready - data store config16 is ready 04:01:55,295 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$e] 04:01:55,306 PM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config16 04:01:55,308 PM [main] [INFO] ShardManagerTest - testOnCreateShardWithLocalMemberNotInShardConfig starting 04:01:55,311 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config17 04:01:55,313 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shard-manager-config17 04:01:55,313 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:01:55,326 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config17: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=foo-ns, moduleName=foo-module, shardName=foo, shardMemberNames=[MemberName{name=member-5}, MemberName{name=member-6}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@696a5e6a] 04:01:55,327 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config17 doCreateShard: shardId: member-1-shard-foo-config17, memberNames: [MemberName{name=member-5}, MemberName{name=member-6}], peerAddresses: {}, isActiveMember: false 04:01:55,329 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config17: findLocalShard : foo 04:01:55,329 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config17: Scheduling 60000 ms timer to wait for shard ShardInformation [shardId=member-1-shard-foo-config17, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=false] 04:01:55,330 PM [test-shard-dispatcher-14] [INFO] Shard - Shard created : member-1-shard-foo-config17, persistent : true 04:01:55,331 PM [test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-1-shard-foo-config17 with journal batch size 1 04:01:55,332 PM [test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$i/member-1-shard-foo-config17/member-1-shard-foo-config17-notifier#19524922 created and ready for shard:member-1-shard-foo-config17 04:01:55,333 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$i/member-1-shard-foo-config17/shard-member-1:snapshot-read#2016710664] 04:01:55,333 PM [test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-foo-config17: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 04:01:55,334 PM [test-shard-dispatcher-14] [INFO] Shard - member-1-shard-foo-config17: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 04:01:55,334 PM [test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-foo-config17: onLeadershipChange, isLeader: false, hasLeader : false 04:01:55,334 PM [test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-foo-config17: onStateChanged: Closing all transaction chains because shard member-1-shard-foo-config17 is no longer the leader 04:01:55,334 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config17: Initializing shard [foo] 04:01:55,334 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-foo-config17 is initialized 04:01:55,334 PM [test-shard-dispatcher-14] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 04:01:55,334 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-foo-config17 is initialized - notifying 1 OnShardInitialized callbacks 04:01:55,334 PM [test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config17 , received role change from null to Follower 04:01:55,334 PM [main] [INFO] ShardManagerTest - testOnCreateShardWithLocalMemberNotInShardConfig ending 04:01:55,334 PM [test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config17 , registered listener akka://test/user/$i 04:01:55,334 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config17: Received role changed for member-1-shard-foo-config17 from null to Follower 04:01:55,436 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$i#-1368283351] 04:01:55,447 PM [test-shard-dispatcher-14] [INFO] Shard - Stopping Shard member-1-shard-foo-config17 04:01:55,447 PM [test-shard-dispatcher-14] [DEBUG] MessageAssembler - member-1-shard-foo-config17 (Follower): Closing 04:01:55,447 PM [test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-foo-config17: Aborting 0 pending queued transactions 04:01:55,448 PM [test-akka.actor.default-dispatcher-15] [INFO] ShardManager - Stopping ShardManager shard-manager-config17 04:01:55,450 PM [main] [INFO] ShardManagerTest - testServerRemovedShardActorNotRunning starting 04:01:55,453 PM [test-akka.actor.default-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-config18 04:01:55,455 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config18 04:01:55,455 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config18: Creating local shard: member-1-shard-default-config18 04:01:55,455 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config18: Creating local shard: member-1-shard-people-config18 04:01:55,465 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config18: findLocalShard : people 04:01:55,465 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config18: Returning NotInitializedException for shard people 04:01:55,466 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config18: findLocalShard : default 04:01:55,466 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config18: Returning NotInitializedException for shard default 04:01:55,467 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config18 : Local Shard replica for shard default has been removed 04:01:55,467 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config18: persisting the shard list [people] 04:01:55,468 PM [main] [INFO] ShardManagerTest - testServerRemovedShardActorNotRunning ending 04:01:55,468 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config18 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 04:01:55,469 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config18: Successfully deleted prior snapshots 04:01:55,569 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$f] 04:01:55,580 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config18 04:01:55,586 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config19 04:01:55,587 PM [test-akka.actor.default-dispatcher-15] [INFO] ShardManager - Recovery complete : shard-manager-config19 04:01:55,587 PM [test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config19: Creating local shard: member-1-shard-default-config19 04:01:55,596 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:01:55,596 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config19 04:01:55,596 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config19: Initializing shard [default] 04:01:55,596 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config19 is initialized 04:01:55,597 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config19: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 04:01:55,597 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config19: Returning NoShardLeaderException for shard default 04:01:55,698 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$j#314356348] 04:01:55,709 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config19 04:01:55,711 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForFollowerShardWithNoInitialLeaderId starting 04:01:55,714 PM [test-akka.actor.default-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-config20 04:01:55,715 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config20 04:01:55,715 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config20: Creating local shard: member-1-shard-default-config20 04:01:55,725 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:01:55,725 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config20 04:01:55,725 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config20: Initializing shard [default] 04:01:55,725 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Shard member-1-shard-default-config20 is initialized 04:01:55,725 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config20: Received role changed for member-1-shard-default-config20 from Candidate to Follower 04:01:55,725 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config20: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 04:01:55,725 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config20: Returning NoShardLeaderException for shard default 04:01:55,725 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config20: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config20, leaderId=member-1-shard-default-config20, leaderPayloadVersion=13] 04:01:55,726 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config20: All Shards are ready - data store config20 is ready 04:01:55,726 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config20: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 04:01:55,726 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config20: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://test@127.0.0.1:2565/user/member-1-shard-default-config#768964047, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$mziea4nU@2d296418] 04:01:55,726 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForFollowerShardWithNoInitialLeaderId starting 04:01:55,827 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$k#1522860491] 04:01:55,838 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config20 04:01:55,844 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config21 04:01:55,845 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config21 04:01:55,865 PM [test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:01:55,865 PM [test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config21: onAddShardReplica: AddShardReplica[ShardName=astronauts] 04:01:55,865 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config21: Sending AddServer message to peer akka://test@127.0.0.1:2565/system/testProbe-45#-1102116264 for shard member-1-shard-astronauts-config21 04:01:55,876 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config21: onAddShardReplica: AddShardReplica[ShardName=astronauts] 04:01:55,877 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config21: A shard replica operation for astronauts is already in progress 04:01:55,981 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithCandidateShard starting 04:01:55,984 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config22 04:01:55,988 PM [test-akka.actor.default-dispatcher-15] [INFO] ShardManager - Recovery complete : shard-manager-config22 04:01:55,988 PM [test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config22: Creating local shard: member-1-shard-default-config22 04:01:55,994 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:01:55,994 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config22 04:01:55,995 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config22: Initializing shard [default] 04:01:55,995 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config22 is initialized 04:01:55,995 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config22: Received role changed for member-1-shard-default-config22 from null to Candidate 04:01:55,995 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config22: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 04:01:55,995 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config22: Scheduling 1200 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config22, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Candidate, leaderId=null, activeMember=true] 04:01:57,208 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config22: Received ShardNotInitializedTimeout message for shard default 04:01:57,208 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config22: Returning NoShardLeaderException for shard default 04:01:57,209 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithCandidateShard ending 04:01:57,311 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$l#-1554783760] 04:01:57,322 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config22 04:01:57,325 PM [main] [INFO] ShardManagerTest - testAddShardReplicaWithAddServerReplyFailure starting 04:01:57,330 PM [test-akka.actor.default-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-config23 04:01:57,331 PM [test-akka.actor.default-dispatcher-15] [INFO] ShardManager - Recovery complete : shard-manager-config23 04:01:57,351 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:01:57,361 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config23: onAddShardReplica: AddShardReplica[ShardName=astronauts] 04:01:57,362 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config23: Sending AddServer message to peer akka://test@127.0.0.1:2565/system/testProbe-50#-47078898 for shard member-1-shard-astronauts-config23 04:01:57,363 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config23: Received AddServerReply{status=TIMEOUT} for shard astronauts from leader akka://test@127.0.0.1:2565/system/testProbe-50#-47078898 04:01:57,363 PM [test-akka.actor.default-dispatcher-5] [WARN] ShardManager - shard-manager-config23: Leader failed to add shard replica astronauts with status TIMEOUT 04:01:57,364 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config23: findLocalShard : astronauts 04:01:57,364 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config23: Local shard astronauts not found - shards present: [] 04:01:57,364 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config23: onAddShardReplica: AddShardReplica[ShardName=astronauts] 04:01:57,365 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config23: Sending AddServer message to peer akka://test@127.0.0.1:2565/system/testProbe-50#-47078898 for shard member-1-shard-astronauts-config23 04:01:57,365 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config23: Received AddServerReply{status=NO_LEADER} for shard astronauts from leader akka://test@127.0.0.1:2565/system/testProbe-50#-47078898 04:01:57,365 PM [test-akka.actor.default-dispatcher-10] [WARN] ShardManager - shard-manager-config23: Leader failed to add shard replica astronauts with status NO_LEADER 04:01:57,366 PM [main] [INFO] ShardManagerTest - testAddShardReplicaWithAddServerReplyFailure ending 04:01:57,467 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-astronauts-config#1797467135] 04:01:57,478 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/shardmanager-config23] 04:01:57,478 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config23 04:01:57,484 PM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config24 04:01:57,486 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config24 04:01:57,486 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config24: Creating local shard: member-1-shard-default-config24 04:01:57,486 PM [main] [INFO] ShardManager - shard-manager-config24: Received role changed for member-1-shard-default-config24 from Candidate to Follower 04:01:57,486 PM [main] [INFO] ShardManager - shard-manager-config24 Received follower initial sync status for member-1-shard-default-config24 status sync done true 04:01:57,486 PM [main] [INFO] ShardManager - shard-manager-config24 Received follower initial sync status for member-1-shard-default-config24 status sync done false 04:01:57,588 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$g] 04:01:57,599 PM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config24 04:01:57,604 PM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config25 04:01:57,605 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config25 04:01:57,605 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config25: Creating local shard: member-1-shard-default-config25 04:01:57,605 PM [main] [INFO] ShardManager - shard-manager-config25: Received role changed for unknown from Candidate to Leader 04:01:57,707 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$h] 04:01:57,718 PM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config25 04:01:57,720 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithUninitializedShard starting 04:01:57,723 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config26 04:01:57,724 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config26 04:01:57,724 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config26: Creating local shard: member-1-shard-default-config26 04:01:57,734 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:01:57,734 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config26 04:01:57,734 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config26: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 04:01:57,734 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config26: Scheduling 600 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config26, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 04:01:58,348 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config26: Received ShardNotInitializedTimeout message for shard default 04:01:58,348 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config26: Returning NotInitializedException for shard default 04:01:58,349 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config26: Initializing shard [default] 04:01:58,349 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config26 is initialized 04:01:58,549 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithUninitializedShard ending 04:01:58,651 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$m#-1944793149] 04:01:58,662 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config26 04:01:58,668 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config27 04:01:58,669 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config27 04:01:58,669 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config27: Creating local shard: member-1-shard-default-config27 04:01:58,679 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:01:58,679 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config27 04:01:58,679 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config27: Initializing shard [default] 04:01:58,679 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Shard member-1-shard-default-config27 is initialized 04:01:58,679 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config27: findLocalShard : default 04:01:58,781 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$n#958088305] 04:01:58,792 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config27 04:01:58,795 PM [main] [INFO] ShardManagerTest - testRegisterForShardLeaderChanges starting 04:01:58,805 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config28 04:01:58,807 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config28 04:01:58,807 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config28: Creating local shard: member-1-shard-default-config28 04:01:58,816 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:01:58,816 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config28 04:01:58,816 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config28: Initializing shard [default] 04:01:58,816 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Shard member-1-shard-default-config28 is initialized 04:01:58,891 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config28: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3c4b03c8 04:01:58,898 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config28: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config28, leaderId=member-1-shard-default-config28, leaderPayloadVersion=13] 04:01:58,904 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config28: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config28, leaderId=member-1-shard-default-config28, leaderPayloadVersion=13] 04:01:59,405 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config28: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config28, leaderId=null, leaderPayloadVersion=13] 04:01:59,416 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config28: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config28, leaderId=member-2-shard-default-config28, leaderPayloadVersion=13] 04:01:59,427 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config28: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config28, leaderId=member-1-shard-default-config28, leaderPayloadVersion=13] 04:01:59,927 PM [main] [INFO] ShardManagerTest - testRegisterForShardLeaderChanges ending 04:02:00,28 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$o#28011224] 04:02:00,39 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config28 04:02:00,45 PM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config29 04:02:00,48 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config29 04:02:00,48 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config29: Creating local shard: member-1-shard-default-config29 04:02:00,150 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$i] 04:02:00,161 PM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config29 04:02:00,167 PM [test-akka.actor.default-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-config30 04:02:00,168 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config30 04:02:00,168 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config30: Creating local shard: member-1-shard-default-config30 04:02:00,177 PM [test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:02:00,177 PM [test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config30 04:02:00,177 PM [test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config30: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=non-existent, waitUntilReady=false] 04:02:00,178 PM [test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config30: No shard found for non-existent 04:02:00,279 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$p#1895471320] 04:02:00,290 PM [test-akka.actor.default-dispatcher-15] [INFO] ShardManager - Stopping ShardManager shard-manager-config30 04:02:00,295 PM [test-akka.actor.default-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-config31 04:02:00,296 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shard-manager-config31 04:02:00,305 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config31: onAddShardReplica: AddShardReplica[ShardName=model-inventory] 04:02:00,305 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config31: No module configuration exists for shard model-inventory 04:02:00,407 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$q#-1232061984] 04:02:00,418 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config31 04:02:00,420 PM [main] [INFO] ShardManagerTest - testAddShardReplicaWithPreExistingReplicaInRemoteShardLeader starting 04:02:00,422 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config32 04:02:00,424 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shard-manager-config32 04:02:00,424 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config32: Creating local shard: member-1-shard-default-config32 04:02:00,433 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:02:00,433 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config32 04:02:00,433 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config32: Initializing shard [default] 04:02:00,433 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Shard member-1-shard-default-config32 is initialized 04:02:00,433 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config32: Received MemberUp: memberName: MemberName{name=leader-member}, address: akka://test 04:02:00,433 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardInformation - updatePeerAddress for peer leader-member-shard-default-config32 with address akka://test/user/shardmanager-config32/leader-member-shard-default-config32 04:02:00,433 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer leader-member-shard-default-config32 with address akka://test/user/shardmanager-config32/leader-member-shard-default-config32 to akka://test/user/member-1-shard-default-config 04:02:00,434 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config32: Received role changed for member-1-shard-default-config32 from Candidate to Follower 04:02:00,434 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config32: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config32, leaderId=leader-member-shard-default-config32, leaderPayloadVersion=13] 04:02:00,434 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config32: All Shards are ready - data store config32 is ready 04:02:00,434 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config32: onAddShardReplica: AddShardReplica[ShardName=default] 04:02:00,434 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config32: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 04:02:00,434 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config32: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://test/user/shardmanager-config32/leader-member-shard-default-config32, primaryVersion=13] 04:02:00,434 PM [test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config32: Sending AddServer message to peer akka://test/user/shardmanager-config32/leader-member-shard-default-config32 for shard member-1-shard-default-config32 04:02:00,435 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config32: Received AddServerReply{status=ALREADY_EXISTS} for shard default from leader akka://test/user/shardmanager-config32/leader-member-shard-default-config32 04:02:00,435 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config32: Local shard default already exists 04:02:00,484 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config32: findLocalShard : default 04:02:00,484 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config32: onAddShardReplica: AddShardReplica[ShardName=default] 04:02:00,485 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config32: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 04:02:00,485 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config32: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://test/user/shardmanager-config32/leader-member-shard-default-config32, primaryVersion=13] 04:02:00,485 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config32: Sending AddServer message to peer akka://test/user/shardmanager-config32/leader-member-shard-default-config32 for shard member-1-shard-default-config32 04:02:00,486 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config32: Received AddServerReply{status=ALREADY_EXISTS} for shard default from leader akka://test/user/shardmanager-config32/leader-member-shard-default-config32 04:02:00,486 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config32: Local shard default already exists 04:02:00,487 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-default-config32 04:02:00,487 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config32: onAddShardReplica: AddShardReplica[ShardName=default] 04:02:00,487 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config32: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 04:02:00,487 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config32: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://test/user/shardmanager-config32/leader-member-shard-default-config32, primaryVersion=13] 04:02:00,488 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config32: Sending AddServer message to peer akka://test/user/shardmanager-config32/leader-member-shard-default-config32 for shard member-1-shard-default-config32 04:02:00,598 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config32: AddServer request to akka://test/user/shardmanager-config32/leader-member-shard-default-config32 for default failed akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://test/), Path(/user/shardmanager-config32/leader-member-shard-default-config32)]] after [100 ms]. Message of type [org.opendaylight.controller.cluster.raft.messages.AddServer]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 04:02:00,602 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config32: findLocalShard : default 04:02:00,602 PM [main] [INFO] ShardManagerTest - testAddShardReplicaWithPreExistingReplicaInRemoteShardLeader ending 04:02:00,704 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/shardmanager-config32] 04:02:00,715 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config32 04:02:00,718 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForRemoteShard starting 04:02:00,726 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 04:02:00,736 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-3224771909218084811] 04:02:00,736 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 04:02:00,748 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 04:02:00,748 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 04:02:00,748 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 04:02:00,748 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 04:02:00,762 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 04:02:00,762 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 04:02:00,762 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 04:02:00,766 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config33] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 04:02:00,768 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Starting ShardManager shard-manager-config33 04:02:00,773 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 04:02:00,773 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config33 04:02:00,774 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config33: Creating local shard: member-1-shard-default-config33 04:02:00,776 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config33: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 04:02:00,776 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-default-config33 04:02:00,794 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [8225109381720987400] 04:02:00,794 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 04:02:00,796 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 04:02:00,797 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 04:02:00,806 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 04:02:00,806 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 04:02:00,814 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config33] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 04:02:00,816 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ShardManager - Starting ShardManager shard-manager-config33 04:02:00,823 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:02:00,823 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config33 04:02:00,823 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-default-config33 04:02:00,823 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-default-config33 to akka://test/user/member-1-shard-default-config 04:02:00,823 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config33: updated peer member-1-shard-default-config33 on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-default-config33 on shard member-1-shard-default-config33 whose actor address is Actor[akka://test/user/member-1-shard-default-config#-192459908] 04:02:00,824 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config33 04:02:00,824 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config33: Creating local shard: member-2-shard-default-config33 04:02:00,825 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config33: Creating local shard: member-2-shard-astronauts-config33 04:02:00,825 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:02:00,825 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-default-config33 04:02:00,825 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-astronauts-config33 04:02:00,825 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config33: Initializing shard [astronauts] 04:02:00,825 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-astronauts-config33 is initialized 04:02:00,825 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config33: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-astronauts-config33, leaderId=member-2-shard-astronauts-config33, leaderPayloadVersion=12] 04:02:00,825 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config33: Received role changed for member-2-shard-astronauts-config33 from Candidate to Leader 04:02:00,856 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 04:02:00,864 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 04:02:00,865 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ShardManager - shard-manager-config33: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 04:02:00,865 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-default-config33 04:02:00,865 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-default-config33 to akka://cluster-test/user/member-2-shard-astronauts-config 04:02:00,865 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-astronauts-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-astronauts-config33 04:02:00,865 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-astronauts-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-astronauts-config33 to akka://cluster-test/user/member-2-shard-astronauts-config 04:02:01,764 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 04:02:01,765 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config33: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 04:02:01,765 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config33 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33/member-2-shard-default-config33 04:02:01,765 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config33 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33/member-2-shard-default-config33 to akka://test/user/member-1-shard-default-config 04:02:01,766 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config33: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=astronauts, waitUntilReady=false] 04:02:01,766 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config33: findPrimary for astronauts forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33, visitedAddresses: [akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33] 04:02:01,774 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config33: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=astronauts, waitUntilReady=false] 04:02:01,774 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config33: Found primary for astronauts: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/member-2-shard-astronauts-config#-1388756626, primaryVersion=12] 04:02:01,777 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForRemoteShard ending 04:02:01,826 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ShardManager - shard-manager-config33: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 04:02:01,826 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config33 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33/member-2-shard-default-config33 04:02:01,826 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config33 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33/member-2-shard-default-config33 to akka://cluster-test/user/member-2-shard-astronauts-config 04:02:01,827 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-astronauts-config33 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33/member-2-shard-astronauts-config33 04:02:01,827 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-astronauts-config33 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33/member-2-shard-astronauts-config33 to akka://cluster-test/user/member-2-shard-astronauts-config 04:02:01,880 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Stopping ShardManager shard-manager-config33 04:02:01,882 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 04:02:01,883 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 04:02:01,895 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 04:02:01,895 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 04:02:01,900 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 04:02:01,924 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ShardManager - Stopping ShardManager shard-manager-config33 04:02:01,926 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 04:02:01,926 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 04:02:02,941 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [01/16/2024 16:02:02.942] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 04:02:02,964 PM [test-akka.actor.default-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-config34 04:02:02,966 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shard-manager-config34 04:02:02,966 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config34: Creating local shard: member-1-shard-default-config34 04:02:02,974 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config34: findLocalShard : default 04:02:02,974 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config34: Returning NotInitializedException for shard default 04:02:03,76 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$r#-1050035117] 04:02:03,87 PM [test-akka.actor.default-dispatcher-17] [INFO] ShardManager - Stopping ShardManager shard-manager-config34 04:02:03,93 PM [test-akka.actor.default-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-config35 04:02:03,96 PM [test-akka.actor.default-dispatcher-17] [INFO] ShardManager - Recovery complete : shard-manager-config35 04:02:03,96 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config35: Creating local shard: member-1-shard-default-config35 04:02:03,102 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:02:03,102 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config35 04:02:03,102 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config35: Initializing shard [default] 04:02:03,102 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-default-config35 is initialized 04:02:03,102 PM [test-akka.actor.default-dispatcher-17] [INFO] ShardManager - shard-manager-config35: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config35, leaderId=member-1-shard-default-config35, leaderPayloadVersion=13] 04:02:03,103 PM [test-akka.actor.default-dispatcher-17] [INFO] ShardManager - shard-manager-config35: Received role changed for member-1-shard-default-config35 from Candidate to Leader 04:02:03,103 PM [test-akka.actor.default-dispatcher-17] [INFO] ShardManager - shard-manager-config35: All Shards are ready - data store config35 is ready 04:02:03,103 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config35: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=default, meberVotingStatusMap={member-2=true}] 04:02:03,103 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config35: findLocalShard : default 04:02:03,103 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config35: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-default-config35=true}, serversVisited=[]] to local shard akka://test/user/member-1-shard-default-config35 04:02:03,103 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config35: Received ServerChangeReply{status=OK} from local shard akka://test/user/member-1-shard-default-config35 04:02:03,103 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config35: ChangeServersVotingStatus succeeded for shard default 04:02:03,254 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-default-config35#-1351895130] 04:02:03,266 PM [main] [INFO] ShardManagerTest - testShutDown starting 04:02:03,269 PM [test-akka.actor.default-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-config36 04:02:03,270 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shard-manager-config36 04:02:03,270 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config36: Creating local shard: member-1-shard-shard2-config36 04:02:03,271 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config36: Creating local shard: member-1-shard-shard1-config36 04:02:03,280 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:02:03,280 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-shard2-config36 04:02:03,280 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-shard1-config36 04:02:03,280 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config36: Initializing shard [shard1] 04:02:03,280 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-shard1-config36 is initialized 04:02:03,280 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config36: Initializing shard [shard2] 04:02:03,280 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-shard2-config36 is initialized 04:02:03,280 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config36: Issuing gracefulStop to shard member-1-shard-shard2-config36 04:02:03,280 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config36: Issuing gracefulStop to shard member-1-shard-shard1-config36 04:02:03,280 PM [test-akka.actor.default-dispatcher-17] [INFO] ShardManager - Shutting down ShardManager shard-manager-config36 - waiting on 2 shards 04:02:03,830 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-shard1-config36#-1657359618] 04:02:03,831 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-shard2-config36#279056817] 04:02:03,831 PM [test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config36: All shards shutdown - sending PoisonPill to self 04:02:03,831 PM [test-akka.actor.default-dispatcher-15] [INFO] ShardManager - Stopping ShardManager shard-manager-config36 04:02:03,832 PM [main] [INFO] ShardManagerTest - testShutDown ending 04:02:03,933 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$t#-1469399402] 04:02:03,945 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForNonLocalLeaderShardBeforeMemberUp starting 04:02:03,947 PM [test-akka.actor.default-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-config37 04:02:03,949 PM [test-akka.actor.default-dispatcher-15] [INFO] ShardManager - Recovery complete : shard-manager-config37 04:02:03,949 PM [test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config37: Creating local shard: member-1-shard-default-config37 04:02:03,957 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:02:03,957 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config37 04:02:03,957 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config37: Initializing shard [default] 04:02:03,957 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Shard member-1-shard-default-config37 is initialized 04:02:03,957 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config37: Received role changed for member-1-shard-default-config37 from Candidate to Follower 04:02:03,957 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManagerTest$TestShardManager - Received unhandled message LeaderStateChanged [memberId=member-1-shard-default-config37, leaderId=member-2-shard-default-config37, leaderPayloadVersion=13] 04:02:03,958 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config37: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 04:02:03,958 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config37: Returning NoShardLeaderException for shard default 04:02:03,958 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForNonLocalLeaderShardBeforeMemberUp ending 04:02:04,59 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$u#1765688074] 04:02:04,70 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config37 04:02:04,71 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithNoRoleShard starting 04:02:04,76 PM [test-akka.actor.default-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-config38 04:02:04,77 PM [test-akka.actor.default-dispatcher-15] [INFO] ShardManager - Recovery complete : shard-manager-config38 04:02:04,77 PM [test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config38: Creating local shard: member-1-shard-default-config38 04:02:04,86 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:02:04,86 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config38 04:02:04,86 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config38: Initializing shard [default] 04:02:04,86 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Shard member-1-shard-default-config38 is initialized 04:02:04,87 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config38: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 04:02:04,87 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config38: Scheduling 1200 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config38, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 04:02:05,298 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config38: Received ShardNotInitializedTimeout message for shard default 04:02:05,298 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config38: Returning NoShardLeaderException for shard default 04:02:05,299 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithNoRoleShard ending 04:02:05,400 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$v#-80509687] 04:02:05,411 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config38 04:02:05,412 PM [main] [INFO] ShardManagerTest - testShardPersistenceWithRestoredData starting 04:02:05,415 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config39 04:02:05,415 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config39: onSnapshotOffer: ShardManagerSnapshot [ShardList = [default, astronauts] ] 04:02:05,415 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config39: adding shard astronauts 04:02:05,416 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config39: removing shard people 04:02:05,416 PM [test-akka.actor.default-dispatcher-15] [INFO] ShardManager - Recovery complete : shard-manager-config39 04:02:05,416 PM [test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config39: Creating local shard: member-1-shard-default-config39 04:02:05,416 PM [test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config39: Creating local shard: member-1-shard-astronauts-config39 04:02:05,427 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config39: findLocalShard : people 04:02:05,427 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config39: Local shard people not found - shards present: [default, astronauts] 04:02:05,427 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config39: findLocalShard : default 04:02:05,427 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config39: Returning NotInitializedException for shard default 04:02:05,428 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config39: findLocalShard : astronauts 04:02:05,428 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config39: Returning NotInitializedException for shard astronauts 04:02:05,428 PM [main] [INFO] ShardManagerTest - testShardPersistenceWithRestoredData ending 04:02:05,529 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$j] 04:02:05,539 PM [test-akka.actor.default-dispatcher-17] [INFO] ShardManager - Stopping ShardManager shard-manager-config39 04:02:05,540 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForNonLocalLeaderShard starting 04:02:05,550 PM [test-akka.actor.default-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-config40 04:02:05,551 PM [test-akka.actor.default-dispatcher-17] [INFO] ShardManager - Recovery complete : shard-manager-config40 04:02:05,551 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config40: Creating local shard: member-1-shard-default-config40 04:02:05,561 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:02:05,561 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config40 04:02:05,561 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config40: Initializing shard [default] 04:02:05,561 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Shard member-1-shard-default-config40 is initialized 04:02:05,561 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config40: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://test 04:02:05,561 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config40 with address akka://test/user/shardmanager-config40/member-2-shard-default-config40 04:02:05,561 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config40 with address akka://test/user/shardmanager-config40/member-2-shard-default-config40 to akka://test/user/member-1-shard-default-config 04:02:05,561 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config40: Received role changed for member-1-shard-default-config40 from Candidate to Follower 04:02:05,561 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config40: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config40, leaderId=member-2-shard-default-config40, leaderPayloadVersion=12] 04:02:05,561 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config40: All Shards are ready - data store config40 is ready 04:02:05,562 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config40: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 04:02:05,562 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config40: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://test/user/shardmanager-config40/member-2-shard-default-config40, primaryVersion=12] 04:02:05,562 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForNonLocalLeaderShard ending 04:02:05,663 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$w#-132543304] 04:02:05,674 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config40 04:02:05,676 PM [main] [INFO] ShardManagerTest - testShardAvailabilityOnChangeOfMemberReachability starting 04:02:05,682 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 04:02:05,692 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [3383067686379957515] 04:02:05,692 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 04:02:05,704 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 04:02:05,704 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 04:02:05,704 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 04:02:05,704 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 04:02:05,704 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 04:02:05,706 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 04:02:05,706 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 04:02:05,709 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config41 04:02:05,710 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config41] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 04:02:05,715 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 04:02:05,715 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shard-manager-config41 04:02:05,715 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config41: Creating local shard: member-1-shard-default-config41 04:02:05,716 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config41: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 04:02:05,716 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-1-shard-default-config41 04:02:05,726 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [8870002559733175657] 04:02:05,727 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 04:02:05,729 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 04:02:05,729 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 04:02:05,749 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 04:02:05,749 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 04:02:05,751 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config41] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 04:02:05,753 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config41 04:02:05,764 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:02:05,764 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config41 04:02:05,764 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-1-shard-default-config41 04:02:05,764 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-1-shard-default-config41 to akka://cluster-test/user/member-1-shard-default-config 04:02:05,764 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config41: updated peer member-1-shard-default-config41 on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-1-shard-default-config41 on shard member-1-shard-default-config41 whose actor address is Actor[akka://cluster-test/user/member-1-shard-default-config#-61190370] 04:02:05,764 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config41: Initializing shard [default] 04:02:05,764 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-default-config41 is initialized 04:02:05,764 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config41: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config41, leaderId=member-2-shard-default-config41, leaderPayloadVersion=13] 04:02:05,764 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config41: Received role changed for member-1-shard-default-config41 from Candidate to Follower 04:02:05,770 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config41 04:02:05,770 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config41: Creating local shard: member-2-shard-default-config41 04:02:05,771 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:02:05,771 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-default-config41 04:02:05,771 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config41: Initializing shard [default] 04:02:05,771 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-default-config41 is initialized 04:02:05,771 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config41: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-config41, leaderId=member-2-shard-default-config41, leaderPayloadVersion=13] 04:02:05,771 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config41: Received role changed for member-2-shard-default-config41 from Candidate to Leader 04:02:05,771 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config41: All Shards are ready - data store config41 is ready 04:02:05,836 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 04:02:05,844 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config41: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 04:02:05,844 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-1-shard-default-config41 04:02:05,844 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-1-shard-default-config41 to akka://cluster-test/user/member-2-shard-default-config 04:02:05,844 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config41: All Shards are ready - data store config41 is ready 04:02:05,844 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 04:02:06,721 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 04:02:06,726 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config41: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 04:02:06,726 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config41/member-2-shard-default-config41 04:02:06,726 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config41/member-2-shard-default-config41 to akka://cluster-test/user/member-1-shard-default-config 04:02:06,726 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config41: All Shards are ready - data store config41 is ready 04:02:06,726 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config41: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 04:02:06,727 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config41: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config41/member-2-shard-default-config41, primaryVersion=13] 04:02:06,727 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 04:02:06,727 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - Marking Leader member-2-shard-default-config41 as unavailable. 04:02:06,727 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config41: Received MemberRemoved: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 04:02:06,727 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config41: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 04:02:06,727 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config41: Scheduling 1200 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config41, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=member-2-shard-default-config41, activeMember=true] 04:02:06,751 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config41: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 04:02:06,751 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config41/member-2-shard-default-config41 04:02:06,751 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config41/member-2-shard-default-config41 to akka://cluster-test/user/member-2-shard-default-config 04:02:06,751 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config41: All Shards are ready - data store config41 is ready 04:02:07,940 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config41: Received ShardNotInitializedTimeout message for shard default 04:02:07,940 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config41: Returning NoShardLeaderException for shard default 04:02:07,941 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Received ReachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 04:02:07,941 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41 04:02:07,941 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41 to akka://cluster-test/user/member-1-shard-default-config 04:02:07,941 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Marking Leader member-2-shard-default-config41 as available. 04:02:07,941 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config41: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 04:02:07,941 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config41: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41, primaryVersion=13] 04:02:07,941 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config41: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 04:02:07,942 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41 04:02:07,942 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41 to akka://cluster-test/user/member-1-shard-default-config 04:02:07,942 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config41: All Shards are ready - data store config41 is ready 04:02:07,942 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 04:02:07,942 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Marking Leader member-2-shard-default-config41 as unavailable. 04:02:07,942 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config41: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 04:02:07,942 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config41: Scheduling 1200 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config41, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=member-2-shard-default-config41, activeMember=true] 04:02:07,942 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Received ReachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 04:02:07,942 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41 04:02:07,942 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41 to akka://cluster-test/user/member-1-shard-default-config 04:02:07,942 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-default-config41 is initialized - notifying 1 OnShardInitialized callbacks 04:02:07,942 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Marking Leader member-2-shard-default-config41 as available. 04:02:07,942 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-default-config41 is ready - notifying 1 OnShardInitialized callbacks 04:02:07,942 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config41: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41, primaryVersion=13] 04:02:07,943 PM [main] [INFO] ShardManagerTest - testShardAvailabilityOnChangeOfMemberReachability ending 04:02:08,45 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-config41 04:02:08,48 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 04:02:08,48 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 04:02:08,56 PM [cluster-test-akka.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 04:02:08,56 PM [cluster-test-akka.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 04:02:08,61 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 04:02:08,72 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config41 04:02:08,73 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 04:02:08,73 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 04:02:09,98 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [01/16/2024 16:02:09.099] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 04:02:09,115 PM [main] [INFO] ShardManagerTest - testAddShardReplica starting 04:02:09,123 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 04:02:09,133 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [2435874614439801549] 04:02:09,133 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 04:02:09,135 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 04:02:09,135 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 04:02:09,150 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 04:02:09,150 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 04:02:09,165 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 04:02:09,165 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 04:02:09,165 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 04:02:09,174 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 04:02:09,175 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config42] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 04:02:09,176 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shardmanager-config42 04:02:09,204 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-5440629766336017079] 04:02:09,204 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 04:02:09,208 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 04:02:09,208 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 04:02:09,215 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shardmanager-config42 04:02:09,215 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shardmanager-config42: Creating local shard: member-1-shard-default-config42 04:02:09,216 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shardmanager-config42: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 04:02:09,216 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-default-config42 04:02:09,218 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 04:02:09,218 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 04:02:09,227 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shardmanager-config42 04:02:09,227 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config42] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 04:02:09,228 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shardmanager-config42 04:02:09,228 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shardmanager-config42: Creating local shard: member-2-shard-default-config42 04:02:09,229 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shardmanager-config42: Creating local shard: member-2-shard-astronauts-config42 04:02:09,234 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:02:09,234 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-2-shard-default-config42 04:02:09,234 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-2-shard-astronauts-config42 04:02:09,234 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shardmanager-config42: Initializing shard [astronauts] 04:02:09,234 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-2-shard-astronauts-config42 is initialized 04:02:09,234 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shardmanager-config42: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-astronauts-config42, leaderId=member-2-shard-astronauts-config42, leaderPayloadVersion=12] 04:02:09,234 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shardmanager-config42: Received role changed for member-2-shard-astronauts-config42 from Candidate to Leader 04:02:09,235 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:02:09,235 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config42 04:02:09,235 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-default-config42 04:02:09,235 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-default-config42 to akka://cluster-test/user/member-1-shard-default-config 04:02:09,235 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shardmanager-config42: updated peer member-1-shard-default-config42 on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-default-config42 on shard member-1-shard-default-config42 whose actor address is Actor[akka://cluster-test/user/member-1-shard-default-config#1243235630] 04:02:09,274 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 04:02:09,282 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 04:02:09,285 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shardmanager-config42: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 04:02:09,285 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-default-config42 04:02:09,285 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-default-config42 to akka://cluster-test/user/member-2-shard-astronauts-config 04:02:09,285 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-astronauts-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-astronauts-config42 04:02:09,285 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-astronauts-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-astronauts-config42 to akka://cluster-test/user/member-2-shard-astronauts-config 04:02:10,151 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 04:02:10,152 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shardmanager-config42: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 04:02:10,152 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config42 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42/member-2-shard-default-config42 04:02:10,152 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config42 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42/member-2-shard-default-config42 to akka://cluster-test/user/member-1-shard-default-config 04:02:10,154 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shardmanager-config42: onAddShardReplica: AddShardReplica[ShardName=astronauts] 04:02:10,155 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shardmanager-config42: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=astronauts, waitUntilReady=true] 04:02:10,155 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shardmanager-config42: findPrimary for astronauts forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42, visitedAddresses: [akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42] 04:02:10,160 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shardmanager-config42: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=astronauts, waitUntilReady=true] 04:02:10,160 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shardmanager-config42: Found primary for astronauts: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/member-2-shard-astronauts-config, primaryVersion=12] 04:02:10,163 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shardmanager-config42: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2559/user/member-2-shard-astronauts-config for shard member-1-shard-astronauts-config42 04:02:10,169 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shardmanager-config42: Received AddServerReply{status=OK, leaderHint=member-2-shard-astronauts-config42} for shard astronauts from leader akka://cluster-test@127.0.0.1:2559/user/member-2-shard-astronauts-config 04:02:10,169 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shardmanager-config42: Leader shard successfully added the replica shard astronauts 04:02:10,169 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-astronauts-config42 04:02:10,169 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shardmanager-config42: persisting the shard list [default, astronauts] 04:02:10,170 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shardmanager-config42 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 04:02:10,170 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shardmanager-config42: Successfully deleted prior snapshots 04:02:10,205 PM [main] [INFO] ShardManagerTest - testAddShardReplica ending 04:02:10,246 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shardmanager-config42: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 04:02:10,246 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config42 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42/member-2-shard-default-config42 04:02:10,246 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config42 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42/member-2-shard-default-config42 to akka://cluster-test/user/member-2-shard-astronauts-config 04:02:10,246 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-astronauts-config42 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42/member-2-shard-astronauts-config42 04:02:10,246 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-astronauts-config42 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42/member-2-shard-astronauts-config42 to akka://cluster-test/user/member-2-shard-astronauts-config 04:02:10,307 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shardmanager-config42 04:02:10,308 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 04:02:10,308 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 04:02:10,313 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 04:02:10,313 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 04:02:10,315 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 04:02:10,331 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shardmanager-config42 04:02:10,332 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 04:02:10,332 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 04:02:11,349 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [01/16/2024 16:02:11.352] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 04:02:11,374 PM [test-akka.actor.default-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-config43 04:02:11,376 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shard-manager-config43 04:02:11,376 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config43: Creating local shard: member-1-shard-default-config43 04:02:11,376 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:02:11,376 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config43 04:02:11,376 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config43: Initializing shard [default] 04:02:11,376 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Shard member-1-shard-default-config43 is initialized 04:02:11,376 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config43: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config43, leaderId=member-1-shard-default-config43, leaderPayloadVersion=13] 04:02:11,377 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config43: Received role changed for member-1-shard-default-config43 from Candidate to Leader 04:02:11,377 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config43: All Shards are ready - data store config43 is ready 04:02:11,377 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config43: onRemoveShardReplica: RemoveShardReplica [shardName=default, memberName=MemberName{name=member-1}] 04:02:11,377 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config43: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 04:02:11,377 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config43: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://test@127.0.0.1:2565/user/member-1-shard-default-config43#-1630749835, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$mziea4nU@611d6bb2] 04:02:11,377 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config43: Sending RemoveServer message to peer akka://test@127.0.0.1:2565/user/member-1-shard-default-config43#-1630749835 for shard member-1-shard-default-config43 04:02:11,378 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config43: Received RemoveServerReply{status=OK} for shard default 04:02:11,378 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config43: Leader shard successfully removed the replica shard default 04:02:11,531 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-default-config43#-1630749835] 04:02:11,535 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithIsolatedLeaderShard starting 04:02:11,538 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config44 04:02:11,539 PM [test-akka.actor.default-dispatcher-15] [INFO] ShardManager - Recovery complete : shard-manager-config44 04:02:11,539 PM [test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config44: Creating local shard: member-1-shard-default-config44 04:02:11,548 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:02:11,548 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config44 04:02:11,548 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config44: Initializing shard [default] 04:02:11,548 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Shard member-1-shard-default-config44 is initialized 04:02:11,548 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config44: Received role changed for member-1-shard-default-config44 from null to IsolatedLeader 04:02:11,549 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config44: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 04:02:11,549 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config44: Scheduling 1200 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config44, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=null, activeMember=true] 04:02:12,768 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config44: Received ShardNotInitializedTimeout message for shard default 04:02:12,768 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config44: Returning NoShardLeaderException for shard default 04:02:12,769 PM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithIsolatedLeaderShard ending 04:02:12,871 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$y#347456098] 04:02:12,882 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config44 04:02:12,886 PM [test-akka.actor.default-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-config45 04:02:12,887 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shard-manager-config45 04:02:12,887 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config45: Creating local shard: member-1-shard-default-config45 04:02:12,897 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:02:12,897 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config45 04:02:12,897 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config45: findLocalShard : non-existent 04:02:12,897 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config45: Local shard non-existent not found - shards present: [default] 04:02:12,998 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$z#-682284425] 04:02:13,9 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config45 04:02:13,14 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config46 04:02:13,15 PM [test-akka.actor.default-dispatcher-17] [INFO] ShardManager - Recovery complete : shard-manager-config46 04:02:13,35 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:02:13,35 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config46: onRemoveShardReplica: RemoveShardReplica [shardName=astronauts, memberName=MemberName{name=member-2}] 04:02:13,36 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config46: Sending RemoveServer message to peer akka://test@127.0.0.1:2565/system/testProbe-92#269157607 for shard member-2-shard-astronauts-config46 04:02:13,47 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config46: onRemoveShardReplica: RemoveShardReplica [shardName=astronauts, memberName=MemberName{name=member-3}] 04:02:13,47 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config46: A shard replica operation for astronauts is already in progress 04:02:13,154 PM [test-akka.actor.default-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-config47 04:02:13,155 PM [test-akka.actor.default-dispatcher-15] [INFO] ShardManager - Recovery complete : shard-manager-config47 04:02:13,156 PM [test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config47: Creating local shard: member-1-shard-default-config47 04:02:13,165 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config47: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 04:02:13,165 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config47: Returning NotInitializedException for shard default 04:02:13,267 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$A#1993253734] 04:02:13,277 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config47 04:02:13,280 PM [main] [INFO] ShardManagerTest - testServerRemovedShardActorRunning starting 04:02:13,285 PM [test-akka.actor.default-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-config48 04:02:13,289 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shard-manager-config48 04:02:13,289 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config48: Creating local shard: member-1-shard-default-config48 04:02:13,289 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config48: Creating local shard: member-1-shard-people-config48 04:02:13,300 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:02:13,300 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config48 04:02:13,300 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config48 04:02:13,301 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config48: Initializing shard [default] 04:02:13,301 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-default-config48 is initialized 04:02:13,301 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config48: findLocalShard : people 04:02:13,301 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config48: Scheduling 600 ms timer to wait for shard ShardInformation [shardId=member-1-shard-people-config48, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 04:02:13,301 PM [test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-people-config48, persistent : true 04:02:13,303 PM [test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-people-config48 with journal batch size 1 04:02:13,305 PM [test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$k/member-1-shard-people-config48/member-1-shard-people-config48-notifier#2037366787 created and ready for shard:member-1-shard-people-config48 04:02:13,305 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$k/member-1-shard-people-config48/shard-member-1:snapshot-read#1946137138] 04:02:13,307 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config48: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 04:02:13,307 PM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config48: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 04:02:13,307 PM [test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config48: Initializing shard [people] 04:02:13,307 PM [test-akka.actor.default-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-people-config48 is initialized 04:02:13,307 PM [test-akka.actor.default-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-people-config48 is initialized - notifying 1 OnShardInitialized callbacks 04:02:13,307 PM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config48: onLeadershipChange, isLeader: false, hasLeader : false 04:02:13,307 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config48: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config48 is no longer the leader 04:02:13,307 PM [test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config48 , received role change from null to Follower 04:02:13,308 PM [test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config48: findLocalShard : default 04:02:13,308 PM [test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config48 , registered listener akka://test/user/$$k 04:02:13,308 PM [test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config48: Received role changed for member-1-shard-people-config48 from null to Follower 04:02:13,308 PM [test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config48 : Sending Shutdown to Shard actor Actor[akka://test/user/member-1-shard-default-config48#-420150901] with 10000 ms timeout 04:02:13,310 PM [test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config48 : Local Shard replica for shard default has been removed 04:02:13,310 PM [test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config48: persisting the shard list [people] 04:02:13,311 PM [test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config48 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 04:02:13,311 PM [test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config48: Successfully deleted prior snapshots 04:02:13,311 PM [main] [INFO] ShardManagerTest - testServerRemovedShardActorRunning ending 04:02:13,412 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-default-config48#-420150901] 04:02:13,423 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config48 : Successfully shut down Shard actor Actor[akka://test/user/member-1-shard-default-config48#-420150901] 04:02:13,423 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$k] 04:02:13,423 PM [test-shard-dispatcher-18] [INFO] Shard - Stopping Shard member-1-shard-people-config48 04:02:13,423 PM [test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-1-shard-people-config48 (Follower): Closing 04:02:13,423 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config48: Aborting 0 pending queued transactions 04:02:13,424 PM [test-akka.actor.default-dispatcher-17] [INFO] ShardManager - Stopping ShardManager shard-manager-config48 04:02:13,427 PM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config49 04:02:13,428 PM [test-akka.actor.default-dispatcher-17] [INFO] ShardManager - Recovery complete : shard-manager-config49 04:02:13,428 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config49: Creating local shard: member-1-shard-default-config49 04:02:13,429 PM [main] [INFO] ShardManager - shard-manager-config49: Received role changed for member-1-shard-default-config49 from null to Follower 04:02:13,429 PM [main] [INFO] ShardManager - shard-manager-config49: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config49, leaderId=member-2-shard-default-config49, leaderPayloadVersion=13] 04:02:13,439 PM [main] [INFO] ShardManager - shard-manager-config49: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://test 04:02:13,439 PM [main] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config49 with address akka://test/user/shardmanager-config49/member-2-shard-default-config49 04:02:13,439 PM [main] [INFO] ShardManager - shard-manager-config49: All Shards are ready - data store config49 is ready 04:02:13,541 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$l] 04:02:13,551 PM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config49 04:02:13,554 PM [main] [INFO] ShardManagerTest - testRestoreFromSnapshot starting 04:02:13,557 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config50 04:02:13,558 PM [test-akka.actor.default-dispatcher-17] [INFO] ShardManager - Recovery complete : shard-manager-config50 04:02:13,558 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config50: Restoring from ShardManagerSnapshot: ShardManagerSnapshot [ShardList = [shard1, shard2, astronauts] ] 04:02:13,558 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config50: onSnapshotOffer: ShardManagerSnapshot [ShardList = [shard1, shard2, astronauts] ] 04:02:13,558 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config50: adding shard shard1 04:02:13,558 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config50: adding shard shard2 04:02:13,558 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config50: adding shard astronauts 04:02:13,559 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config50: Creating local shard: member-1-shard-shard2-config50 04:02:13,559 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config50: Creating local shard: member-1-shard-astronauts-config50 04:02:13,559 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config50: Creating local shard: member-1-shard-shard1-config50 04:02:13,559 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:02:13,559 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-shard2-config50 04:02:13,560 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-astronauts-config50 04:02:13,560 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-shard1-config50 04:02:13,561 PM [test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-shard2-config50, persistent : true 04:02:13,562 PM [test-shard-dispatcher-20] [INFO] Shard - Shard created : member-1-shard-shard1-config50, persistent : true 04:02:13,562 PM [test-shard-dispatcher-19] [INFO] Shard - Shard created : member-1-shard-astronauts-config50, persistent : true 04:02:13,563 PM [test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$m/member-1-shard-astronauts-config50/member-1-shard-astronauts-config50-notifier#-405218026 created and ready for shard:member-1-shard-astronauts-config50 04:02:13,563 PM [test-akka.actor.default-dispatcher-15] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$m/member-1-shard-astronauts-config50/shard-member-1:snapshot-read#711796065] 04:02:13,563 PM [test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-shard2-config50 with journal batch size 1 04:02:13,563 PM [test-shard-dispatcher-19] [INFO] Shard - Starting recovery for member-1-shard-astronauts-config50 with journal batch size 1 04:02:13,563 PM [test-shard-dispatcher-20] [INFO] Shard - Starting recovery for member-1-shard-shard1-config50 with journal batch size 1 04:02:13,564 PM [test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$m/member-1-shard-shard2-config50/member-1-shard-shard2-config50-notifier#-632704052 created and ready for shard:member-1-shard-shard2-config50 04:02:13,564 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$m/member-1-shard-shard2-config50/shard-member-1:snapshot-read#1860846724] 04:02:13,564 PM [test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$m/member-1-shard-shard1-config50/member-1-shard-shard1-config50-notifier#1345710382 created and ready for shard:member-1-shard-shard1-config50 04:02:13,564 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$m/member-1-shard-shard1-config50/shard-member-1:snapshot-read#-85693710] 04:02:13,566 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config50: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 04:02:13,566 PM [test-shard-dispatcher-19] [INFO] Shard - member-1-shard-shard2-config50: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 04:02:13,567 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 04:02:13,567 PM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-astronauts-config50: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 04:02:13,567 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config50: Initializing shard [shard2] 04:02:13,567 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Shard member-1-shard-shard2-config50 is initialized 04:02:13,567 PM [test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard2-config50: onLeadershipChange, isLeader: false, hasLeader : false 04:02:13,567 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config50: onStateChanged: Closing all transaction chains because shard member-1-shard-shard2-config50 is no longer the leader 04:02:13,567 PM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config50: onLeadershipChange, isLeader: false, hasLeader : false 04:02:13,567 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50: onStateChanged: Closing all transaction chains because shard member-1-shard-astronauts-config50 is no longer the leader 04:02:13,567 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config50: Initializing shard [astronauts] 04:02:13,567 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Shard member-1-shard-astronauts-config50 is initialized 04:02:13,567 PM [test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config50 , received role change from null to Follower 04:02:13,567 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50 (Follower): Received TimeoutNow - switching to Candidate 04:02:13,567 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config50 (Follower): Received TimeoutNow - switching to Candidate 04:02:13,567 PM [test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config50 , received role change from null to Follower 04:02:13,567 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50 (Candidate): Election: Candidate has following voting peers: [] 04:02:13,567 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config50 (Candidate): Election: Candidate has following voting peers: [] 04:02:13,567 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50: Set currentTerm=1, votedFor=member-1-shard-astronauts-config50 04:02:13,567 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config50: Set currentTerm=1, votedFor=member-1-shard-shard2-config50 04:02:13,567 PM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-astronauts-config50 (Candidate): Starting new election term 1 04:02:13,567 PM [test-shard-dispatcher-19] [INFO] Shard - member-1-shard-shard2-config50 (Candidate): Starting new election term 1 04:02:13,567 PM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-astronauts-config50 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 04:02:13,567 PM [test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-1-shard-astronauts-config50 (Follower): Closing 04:02:13,567 PM [test-shard-dispatcher-19] [INFO] Shard - member-1-shard-shard2-config50 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 04:02:13,567 PM [test-shard-dispatcher-19] [DEBUG] MessageAssembler - member-1-shard-shard2-config50 (Follower): Closing 04:02:13,567 PM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config50: onLeadershipChange, isLeader: false, hasLeader : false 04:02:13,567 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50: onStateChanged: Closing all transaction chains because shard member-1-shard-astronauts-config50 is no longer the leader 04:02:13,567 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard1-config50: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 04:02:13,567 PM [test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard2-config50: onLeadershipChange, isLeader: false, hasLeader : false 04:02:13,567 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config50: onStateChanged: Closing all transaction chains because shard member-1-shard-shard2-config50 is no longer the leader 04:02:13,567 PM [test-shard-dispatcher-20] [INFO] Shard - member-1-shard-shard1-config50: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 04:02:13,567 PM [test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config50 , received role change from Follower to Candidate 04:02:13,567 PM [test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config50 , received role change from Follower to Candidate 04:02:13,567 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config50: Initializing shard [shard1] 04:02:13,567 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-shard1-config50 is initialized 04:02:13,567 PM [test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard1-config50: onLeadershipChange, isLeader: false, hasLeader : false 04:02:13,567 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard1-config50: onStateChanged: Closing all transaction chains because shard member-1-shard-shard1-config50 is no longer the leader 04:02:13,567 PM [test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config50 , received role change from null to Follower 04:02:13,567 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard1-config50 (Follower): Received TimeoutNow - switching to Candidate 04:02:13,567 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard1-config50 (Candidate): Election: Candidate has following voting peers: [] 04:02:13,567 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard1-config50: Set currentTerm=1, votedFor=member-1-shard-shard1-config50 04:02:13,567 PM [test-shard-dispatcher-20] [INFO] Shard - member-1-shard-shard1-config50 (Candidate): Starting new election term 1 04:02:13,567 PM [test-shard-dispatcher-20] [INFO] Shard - member-1-shard-shard1-config50 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 04:02:13,567 PM [test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-1-shard-shard1-config50 (Follower): Closing 04:02:13,567 PM [test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard1-config50: onLeadershipChange, isLeader: false, hasLeader : false 04:02:13,567 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard1-config50: onStateChanged: Closing all transaction chains because shard member-1-shard-shard1-config50 is no longer the leader 04:02:13,567 PM [test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config50 , received role change from Follower to Candidate 04:02:13,568 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-astronauts-config50 (Candidate): Received ElectionTimeout 04:02:13,568 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-astronauts-config50 (Leader): Election: Leader has following peers: [] 04:02:13,568 PM [test-shard-dispatcher-20] [INFO] Shard - member-1-shard-astronauts-config50 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 04:02:13,568 PM [test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config50: onLeadershipChange, isLeader: true, hasLeader : true 04:02:13,568 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-astronauts-config50: became leader with frontend state for [] 04:02:13,568 PM [test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config50 , received role change from Candidate to Leader 04:02:13,568 PM [test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config50 , registered listener akka://test/user/$$m 04:02:13,568 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config50: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-astronauts-config50, leaderId=member-1-shard-astronauts-config50, leaderPayloadVersion=13] 04:02:13,568 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config50: Received role changed for member-1-shard-astronauts-config50 from Candidate to Leader 04:02:13,568 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard2-config50 (Candidate): Received ElectionTimeout 04:02:13,568 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard1-config50 (Candidate): Received ElectionTimeout 04:02:13,568 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard1-config50 (Leader): Election: Leader has following peers: [] 04:02:13,568 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard2-config50 (Leader): Election: Leader has following peers: [] 04:02:13,568 PM [test-shard-dispatcher-19] [INFO] Shard - member-1-shard-shard1-config50 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 04:02:13,568 PM [test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard1-config50: onLeadershipChange, isLeader: true, hasLeader : true 04:02:13,568 PM [test-shard-dispatcher-20] [INFO] Shard - member-1-shard-shard2-config50 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 04:02:13,568 PM [test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard2-config50: onLeadershipChange, isLeader: true, hasLeader : true 04:02:13,568 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard1-config50: became leader with frontend state for [] 04:02:13,568 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard2-config50: became leader with frontend state for [] 04:02:13,569 PM [test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config50 , received role change from Candidate to Leader 04:02:13,569 PM [test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config50 , registered listener akka://test/user/$$m 04:02:13,569 PM [test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config50: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-shard1-config50, leaderId=member-1-shard-shard1-config50, leaderPayloadVersion=13] 04:02:13,569 PM [test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config50: Received role changed for member-1-shard-shard1-config50 from Candidate to Leader 04:02:13,569 PM [test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config50 , received role change from Candidate to Leader 04:02:13,569 PM [test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config50 , registered listener akka://test/user/$$m 04:02:13,569 PM [test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config50: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-shard2-config50, leaderId=member-1-shard-shard2-config50, leaderPayloadVersion=13] 04:02:13,569 PM [test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config50: Received role changed for member-1-shard-shard2-config50 from Candidate to Leader 04:02:13,569 PM [test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config50: All Shards are ready - data store config50 is ready 04:02:13,570 PM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config50: findLocalShard : shard1 04:02:13,570 PM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config50: findLocalShard : shard2 04:02:13,570 PM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config50: findLocalShard : astronauts 04:02:13,570 PM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config50: onGetSnapshot 04:02:13,571 PM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config50: Expecting 3 shard snapshot replies 04:02:13,571 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard2-config50: onGetSnapshot 04:02:13,571 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config50: onGetSnapshot 04:02:13,571 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config50: onGetSnapshot 04:02:13,571 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config50: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 04:02:13,571 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config50: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 04:02:13,571 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard2-config50: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 04:02:13,577 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config50: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 04:02:13,577 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard2-config50: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 04:02:13,577 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config50: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 04:02:13,577 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config50: Received GetSnapshotReply [id=member-1-shard-shard1-config50, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@42a0de20, electionTerm=1, electionVotedFor=member-1-shard-shard1-config50, ServerConfigPayload=null]] 04:02:13,577 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config50: Received GetSnapshotReply [id=member-1-shard-shard2-config50, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@7f686a0a, electionTerm=1, electionVotedFor=member-1-shard-shard2-config50, ServerConfigPayload=null]] 04:02:13,577 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config50: Received GetSnapshotReply [id=member-1-shard-astronauts-config50, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@708ed408, electionTerm=1, electionVotedFor=member-1-shard-astronauts-config50, ServerConfigPayload=null]] 04:02:13,577 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config50: All shard snapshots received 04:02:13,578 PM [main] [INFO] ShardManagerTest - testRestoreFromSnapshot ending 04:02:13,680 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$m] 04:02:13,691 PM [test-shard-dispatcher-18] [INFO] Shard - Stopping Shard member-1-shard-astronauts-config50 04:02:13,691 PM [test-shard-dispatcher-18] [DEBUG] MessageSlicer - member-1-shard-astronauts-config50 (Leader)_slicer-id-161: Closing 04:02:13,691 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50: Aborting 0 pending queued transactions 04:02:13,691 PM [test-shard-dispatcher-18] [INFO] Shard - Stopping Shard member-1-shard-shard1-config50 04:02:13,691 PM [test-shard-dispatcher-18] [DEBUG] MessageSlicer - member-1-shard-shard1-config50 (Leader)_slicer-id-162: Closing 04:02:13,692 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config50: Aborting 0 pending queued transactions 04:02:13,692 PM [test-shard-dispatcher-18] [INFO] Shard - Stopping Shard member-1-shard-shard2-config50 04:02:13,692 PM [test-shard-dispatcher-18] [DEBUG] MessageSlicer - member-1-shard-shard2-config50 (Leader)_slicer-id-163: Closing 04:02:13,692 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config50: Aborting 0 pending queued transactions 04:02:13,692 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config50 04:02:13,695 PM [main] [INFO] ShardManagerTest - testOnCreateShardWithNoInitialSchemaContext starting 04:02:13,699 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config51 04:02:13,700 PM [test-akka.actor.default-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-config51 04:02:13,710 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config51: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=foo-ns, moduleName=foo-module, shardName=foo, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@fdd500b] 04:02:13,710 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config51 doCreateShard: shardId: member-1-shard-foo-config51, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: true 04:02:13,711 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:02:13,711 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Creating Shard member-1-shard-foo-config51 04:02:13,711 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config51: findLocalShard : foo 04:02:13,711 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config51: Scheduling 600 ms timer to wait for shard ShardInformation [shardId=member-1-shard-foo-config51, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 04:02:13,712 PM [test-shard-dispatcher-20] [INFO] Shard - Shard created : member-1-shard-foo-config51, persistent : true 04:02:13,712 PM [test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$B/member-1-shard-foo-config51/member-1-shard-foo-config51-notifier#1869554573 created and ready for shard:member-1-shard-foo-config51 04:02:13,712 PM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$B/member-1-shard-foo-config51/shard-member-1:snapshot-read#-163921354] 04:02:13,713 PM [test-shard-dispatcher-20] [INFO] Shard - Starting recovery for member-1-shard-foo-config51 with journal batch size 1 04:02:13,714 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-foo-config51: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 04:02:13,714 PM [test-shard-dispatcher-20] [INFO] Shard - member-1-shard-foo-config51: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 04:02:13,715 PM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config51: Initializing shard [foo] 04:02:13,715 PM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-foo-config51 is initialized 04:02:13,715 PM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-foo-config51 is initialized - notifying 1 OnShardInitialized callbacks 04:02:13,715 PM [test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-foo-config51: onLeadershipChange, isLeader: false, hasLeader : false 04:02:13,715 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-foo-config51: onStateChanged: Closing all transaction chains because shard member-1-shard-foo-config51 is no longer the leader 04:02:13,715 PM [test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config51 , received role change from null to Follower 04:02:13,715 PM [main] [INFO] ShardManagerTest - testOnCreateShardWithNoInitialSchemaContext ending 04:02:13,715 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-foo-config51 (Follower): Received TimeoutNow - switching to Candidate 04:02:13,715 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-foo-config51 (Candidate): Election: Candidate has following voting peers: [] 04:02:13,715 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-foo-config51: Set currentTerm=1, votedFor=member-1-shard-foo-config51 04:02:13,715 PM [test-shard-dispatcher-20] [INFO] Shard - member-1-shard-foo-config51 (Candidate): Starting new election term 1 04:02:13,715 PM [test-shard-dispatcher-20] [INFO] Shard - member-1-shard-foo-config51 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 04:02:13,715 PM [test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-1-shard-foo-config51 (Follower): Closing 04:02:13,715 PM [test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-foo-config51: onLeadershipChange, isLeader: false, hasLeader : false 04:02:13,715 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-foo-config51: onStateChanged: Closing all transaction chains because shard member-1-shard-foo-config51 is no longer the leader 04:02:13,715 PM [test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config51 , received role change from Follower to Candidate 04:02:13,716 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-foo-config51 (Candidate): Received ElectionTimeout 04:02:13,716 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-foo-config51 (Leader): Election: Leader has following peers: [] 04:02:13,716 PM [test-shard-dispatcher-20] [INFO] Shard - member-1-shard-foo-config51 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 04:02:13,716 PM [test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-foo-config51: onLeadershipChange, isLeader: true, hasLeader : true 04:02:13,716 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-foo-config51: became leader with frontend state for [] 04:02:13,716 PM [test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config51 , received role change from Candidate to Leader 04:02:13,717 PM [test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config51 , registered listener akka://test/user/$B 04:02:13,717 PM [test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config51: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-foo-config51, leaderId=member-1-shard-foo-config51, leaderPayloadVersion=13] 04:02:13,717 PM [test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config51: Received role changed for member-1-shard-foo-config51 from Candidate to Leader 04:02:13,717 PM [test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config51: All Shards are ready - data store config51 is ready 04:02:13,817 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$B#-1427638591] 04:02:13,828 PM [test-shard-dispatcher-20] [INFO] Shard - Stopping Shard member-1-shard-foo-config51 04:02:13,828 PM [test-shard-dispatcher-20] [DEBUG] MessageSlicer - member-1-shard-foo-config51 (Leader)_slicer-id-165: Closing 04:02:13,828 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-foo-config51: Aborting 0 pending queued transactions 04:02:13,828 PM [test-akka.actor.default-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-config51 04:02:13,830 PM [main] [INFO] ShardManagerTest - testGetSnapshot starting 04:02:13,832 PM [test-akka.actor.default-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-config52 04:02:13,833 PM [test-akka.actor.default-dispatcher-15] [INFO] ShardManager - Recovery complete : shard-manager-config52 04:02:13,833 PM [test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config52: Creating local shard: member-1-shard-shard2-config52 04:02:13,833 PM [test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config52: Creating local shard: member-1-shard-shard1-config52 04:02:13,843 PM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config52: onGetSnapshot 04:02:13,843 PM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:02:13,843 PM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-shard2-config52 04:02:13,843 PM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-shard1-config52 04:02:13,843 PM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config52: findLocalShard : shard1 04:02:13,843 PM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config52: Scheduling 600 ms timer to wait for shard ShardInformation [shardId=member-1-shard-shard1-config52, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 04:02:13,843 PM [test-shard-dispatcher-20] [INFO] Shard - Shard created : member-1-shard-shard2-config52, persistent : true 04:02:13,844 PM [test-shard-dispatcher-20] [INFO] Shard - Starting recovery for member-1-shard-shard2-config52 with journal batch size 1 04:02:13,844 PM [test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$n/member-1-shard-shard2-config52/member-1-shard-shard2-config52-notifier#-1521642691 created and ready for shard:member-1-shard-shard2-config52 04:02:13,844 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$n/member-1-shard-shard2-config52/shard-member-1:snapshot-read#1754171217] 04:02:13,844 PM [test-shard-dispatcher-20] [INFO] Shard - Shard created : member-1-shard-shard1-config52, persistent : true 04:02:13,844 PM [test-shard-dispatcher-20] [INFO] Shard - Starting recovery for member-1-shard-shard1-config52 with journal batch size 1 04:02:13,844 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$n/member-1-shard-shard1-config52/shard-member-1:snapshot-read#-1129502793] 04:02:13,844 PM [test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$n/member-1-shard-shard1-config52/member-1-shard-shard1-config52-notifier#-1940580853 created and ready for shard:member-1-shard-shard1-config52 04:02:13,845 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard2-config52: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 04:02:13,845 PM [test-shard-dispatcher-20] [INFO] Shard - member-1-shard-shard2-config52: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 04:02:13,845 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config52: Initializing shard [shard2] 04:02:13,845 PM [test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard2-config52: onLeadershipChange, isLeader: false, hasLeader : false 04:02:13,845 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-shard2-config52 is initialized 04:02:13,845 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard2-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-shard2-config52 is no longer the leader 04:02:13,845 PM [test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config52 , received role change from null to Follower 04:02:13,845 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard2-config52 (Follower): Received TimeoutNow - switching to Candidate 04:02:13,845 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard2-config52 (Candidate): Election: Candidate has following voting peers: [] 04:02:13,845 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard2-config52: Set currentTerm=1, votedFor=member-1-shard-shard2-config52 04:02:13,845 PM [test-shard-dispatcher-20] [INFO] Shard - member-1-shard-shard2-config52 (Candidate): Starting new election term 1 04:02:13,845 PM [test-shard-dispatcher-20] [INFO] Shard - member-1-shard-shard2-config52 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 04:02:13,845 PM [test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-1-shard-shard2-config52 (Follower): Closing 04:02:13,845 PM [test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard2-config52: onLeadershipChange, isLeader: false, hasLeader : false 04:02:13,845 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard2-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-shard2-config52 is no longer the leader 04:02:13,845 PM [test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config52 , received role change from Follower to Candidate 04:02:13,845 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard2-config52 (Candidate): Received ElectionTimeout 04:02:13,845 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard2-config52 (Leader): Election: Leader has following peers: [] 04:02:13,845 PM [test-shard-dispatcher-20] [INFO] Shard - member-1-shard-shard2-config52 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 04:02:13,846 PM [test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard2-config52: onLeadershipChange, isLeader: true, hasLeader : true 04:02:13,846 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard2-config52: became leader with frontend state for [] 04:02:13,846 PM [test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config52 , received role change from Candidate to Leader 04:02:13,846 PM [test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config52 , registered listener akka://test/user/$$n 04:02:13,846 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config52: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-shard2-config52, leaderId=member-1-shard-shard2-config52, leaderPayloadVersion=13] 04:02:13,846 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config52: Received role changed for member-1-shard-shard2-config52 from Candidate to Leader 04:02:13,846 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard1-config52: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 04:02:13,846 PM [test-shard-dispatcher-20] [INFO] Shard - member-1-shard-shard1-config52: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 04:02:13,846 PM [test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard1-config52: onLeadershipChange, isLeader: false, hasLeader : false 04:02:13,846 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config52: Initializing shard [shard1] 04:02:13,846 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard1-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-shard1-config52 is no longer the leader 04:02:13,846 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-shard1-config52 is initialized 04:02:13,846 PM [test-akka.actor.default-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-shard1-config52 is initialized - notifying 1 OnShardInitialized callbacks 04:02:13,846 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard1-config52 (Follower): Received TimeoutNow - switching to Candidate 04:02:13,846 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard1-config52 (Candidate): Election: Candidate has following voting peers: [] 04:02:13,846 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard1-config52: Set currentTerm=1, votedFor=member-1-shard-shard1-config52 04:02:13,846 PM [test-shard-dispatcher-20] [INFO] Shard - member-1-shard-shard1-config52 (Candidate): Starting new election term 1 04:02:13,846 PM [test-shard-dispatcher-20] [INFO] Shard - member-1-shard-shard1-config52 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 04:02:13,846 PM [test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-1-shard-shard1-config52 (Follower): Closing 04:02:13,846 PM [test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard1-config52: onLeadershipChange, isLeader: false, hasLeader : false 04:02:13,846 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard1-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-shard1-config52 is no longer the leader 04:02:13,846 PM [test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config52 , received role change from null to Follower 04:02:13,846 PM [test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config52 , received role change from Follower to Candidate 04:02:13,846 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config52: findLocalShard : shard2 04:02:13,846 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard1-config52 (Candidate): Received ElectionTimeout 04:02:13,846 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard1-config52 (Leader): Election: Leader has following peers: [] 04:02:13,846 PM [test-shard-dispatcher-20] [INFO] Shard - member-1-shard-shard1-config52 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 04:02:13,846 PM [test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard1-config52: onLeadershipChange, isLeader: true, hasLeader : true 04:02:13,846 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard1-config52: became leader with frontend state for [] 04:02:13,846 PM [test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config52: onGetSnapshot 04:02:13,846 PM [test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config52 , received role change from Candidate to Leader 04:02:13,846 PM [test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config52 , registered listener akka://test/user/$$n 04:02:13,847 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard2-config52: onGetSnapshot 04:02:13,847 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard2-config52: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 04:02:13,847 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Expecting 2 shard snapshot replies 04:02:13,847 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config52: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-shard1-config52, leaderId=member-1-shard-shard1-config52, leaderPayloadVersion=13] 04:02:13,847 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard1-config52: onGetSnapshot 04:02:13,847 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config52: Received role changed for member-1-shard-shard1-config52 from Candidate to Leader 04:02:13,847 PM [test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config52: All Shards are ready - data store config52 is ready 04:02:13,847 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard1-config52: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 04:02:13,847 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard2-config52: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 04:02:13,847 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Received GetSnapshotReply [id=member-1-shard-shard2-config52, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@34cccdb7, electionTerm=1, electionVotedFor=member-1-shard-shard2-config52, ServerConfigPayload=null]] 04:02:13,847 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard1-config52: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 04:02:13,847 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Received GetSnapshotReply [id=member-1-shard-shard1-config52, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@66489343, electionTerm=1, electionVotedFor=member-1-shard-shard1-config52, ServerConfigPayload=null]] 04:02:13,847 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: All shard snapshots received 04:02:13,849 PM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config52: onAddShardReplica: AddShardReplica[ShardName=astronauts] 04:02:13,850 PM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config52: Sending AddServer message to peer akka://test@127.0.0.1:2565/system/testProbe-106#1156850386 for shard member-1-shard-astronauts-config52 04:02:13,850 PM [test-shard-dispatcher-19] [INFO] Shard - Shard created : member-1-shard-astronauts-config52, persistent : true 04:02:13,851 PM [test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$n/member-1-shard-astronauts-config52/member-1-shard-astronauts-config52-notifier#1533115692 created and ready for shard:member-1-shard-astronauts-config52 04:02:13,851 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$n/member-1-shard-astronauts-config52/shard-member-1:snapshot-read#-732122280] 04:02:13,851 PM [test-shard-dispatcher-19] [INFO] Shard - Starting recovery for member-1-shard-astronauts-config52 with journal batch size 1 04:02:13,852 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config52: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 04:02:13,852 PM [test-shard-dispatcher-19] [INFO] Shard - member-1-shard-astronauts-config52: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 04:02:13,852 PM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config52: Initializing shard [astronauts] 04:02:13,852 PM [test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config52: onLeadershipChange, isLeader: false, hasLeader : false 04:02:13,852 PM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-astronauts-config52 is initialized 04:02:13,852 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-astronauts-config52 is no longer the leader 04:02:13,853 PM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config52 , received role change from null to Follower 04:02:13,853 PM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config52: Received AddServerReply{status=OK, leaderHint=} for shard astronauts from leader akka://test@127.0.0.1:2565/system/testProbe-106#1156850386 04:02:13,853 PM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config52: Leader shard successfully added the replica shard astronauts 04:02:13,853 PM [test-shard-dispatcher-19] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 04:02:13,853 PM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config52 , registered listener akka://test/user/$$n 04:02:13,853 PM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-astronauts-config52 04:02:13,853 PM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config52: persisting the shard list [shard2, astronauts, shard1] 04:02:13,853 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config52: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 04:02:13,853 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config52: Re-initializing to Follower with previous leaderId null 04:02:13,853 PM [test-shard-dispatcher-19] [DEBUG] MessageAssembler - member-1-shard-astronauts-config52 (Follower): Closing 04:02:13,853 PM [test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config52: onLeadershipChange, isLeader: false, hasLeader : false 04:02:13,853 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-astronauts-config52 is no longer the leader 04:02:13,853 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config52 (Follower): Received TimeoutNow - switching to Candidate 04:02:13,853 PM [test-akka.actor.default-dispatcher-22] [INFO] ShardManager - shard-manager-config52: Received role changed for member-1-shard-astronauts-config52 from null to Follower 04:02:13,853 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config52 (Candidate): Election: Candidate has following voting peers: [] 04:02:13,853 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config52: Set currentTerm=1, votedFor=member-1-shard-astronauts-config52 04:02:13,853 PM [test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config52 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 04:02:13,853 PM [test-shard-dispatcher-19] [INFO] Shard - member-1-shard-astronauts-config52 (Candidate): Starting new election term 1 04:02:13,853 PM [test-shard-dispatcher-19] [INFO] Shard - member-1-shard-astronauts-config52 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 04:02:13,854 PM [test-shard-dispatcher-19] [DEBUG] MessageAssembler - member-1-shard-astronauts-config52 (Follower): Closing 04:02:13,854 PM [test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config52: onLeadershipChange, isLeader: false, hasLeader : false 04:02:13,854 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-astronauts-config52 is no longer the leader 04:02:13,854 PM [test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config52 , received role change from Follower to Candidate 04:02:13,854 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config52: Successfully deleted prior snapshots 04:02:13,854 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config52: findLocalShard : astronauts 04:02:13,854 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config52 (Candidate): Received ElectionTimeout 04:02:13,854 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config52 (Leader): Election: Leader has following peers: [] 04:02:13,854 PM [test-shard-dispatcher-19] [INFO] Shard - member-1-shard-astronauts-config52 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 04:02:13,854 PM [test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config52: onLeadershipChange, isLeader: true, hasLeader : true 04:02:13,854 PM [test-akka.actor.default-dispatcher-17] [INFO] ShardManager - shard-manager-config52: Received role changed for member-1-shard-astronauts-config52 from Follower to Candidate 04:02:13,854 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config52: became leader with frontend state for [] 04:02:13,854 PM [test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config52 , received role change from Candidate to Leader 04:02:13,854 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config52: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-astronauts-config52, leaderId=member-1-shard-astronauts-config52, leaderPayloadVersion=13] 04:02:13,854 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config52: onGetSnapshot 04:02:13,854 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config52: Received role changed for member-1-shard-astronauts-config52 from Candidate to Leader 04:02:13,854 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config52: All Shards are ready - data store config52 is ready 04:02:13,854 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config52: onGetSnapshot 04:02:13,854 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Expecting 3 shard snapshot replies 04:02:13,854 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config52: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 04:02:13,855 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-astronauts-config52: onGetSnapshot 04:02:13,855 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-astronauts-config52: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 04:02:13,855 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config52: onGetSnapshot 04:02:13,855 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config52: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 04:02:13,855 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-astronauts-config52: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 04:02:13,855 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-shard2-config52: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 04:02:13,855 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Received GetSnapshotReply [id=member-1-shard-shard2-config52, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@1350d81f, electionTerm=1, electionVotedFor=member-1-shard-shard2-config52, ServerConfigPayload=null]] 04:02:13,855 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config52: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 04:02:13,855 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Received GetSnapshotReply [id=member-1-shard-astronauts-config52, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@534ab272, electionTerm=1, electionVotedFor=member-1-shard-astronauts-config52, ServerConfigPayload=null]] 04:02:13,855 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Received GetSnapshotReply [id=member-1-shard-shard1-config52, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@31482193, electionTerm=1, electionVotedFor=member-1-shard-shard1-config52, ServerConfigPayload=null]] 04:02:13,855 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: All shard snapshots received 04:02:13,855 PM [main] [INFO] ShardManagerTest - testGetSnapshot ending 04:02:13,956 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$n] 04:02:13,967 PM [test-shard-dispatcher-20] [INFO] Shard - Stopping Shard member-1-shard-shard1-config52 04:02:13,967 PM [test-shard-dispatcher-20] [DEBUG] MessageSlicer - member-1-shard-shard1-config52 (Leader)_slicer-id-169: Closing 04:02:13,967 PM [test-shard-dispatcher-18] [INFO] Shard - Stopping Shard member-1-shard-shard2-config52 04:02:13,967 PM [test-shard-dispatcher-19] [INFO] Shard - Stopping Shard member-1-shard-astronauts-config52 04:02:13,967 PM [test-shard-dispatcher-18] [DEBUG] MessageSlicer - member-1-shard-shard2-config52 (Leader)_slicer-id-168: Closing 04:02:13,967 PM [test-shard-dispatcher-19] [DEBUG] MessageSlicer - member-1-shard-astronauts-config52 (Leader)_slicer-id-171: Closing 04:02:13,967 PM [test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-shard1-config52: Aborting 0 pending queued transactions 04:02:13,967 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-astronauts-config52: Aborting 0 pending queued transactions 04:02:13,967 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config52: Aborting 0 pending queued transactions 04:02:13,968 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config52 04:02:13,970 PM [main] [INFO] ShardManagerTest - testWhenMultipleShardsPresentSyncStatusMustBeTrueForAllShards starting 04:02:13,973 PM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config53 04:02:13,974 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config53 04:02:13,974 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config53: Creating local shard: member-1-shard-default-config53 04:02:13,975 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config53: Creating local shard: member-1-shard-astronauts-config53 04:02:13,975 PM [main] [INFO] ShardManager - shard-manager-config53: Received role changed for member-1-shard-default-config53 from Follower to Leader 04:02:13,975 PM [main] [INFO] ShardManager - shard-manager-config53: Received role changed for member-1-shard-astronauts-config53 from Follower to Leader 04:02:13,975 PM [main] [INFO] ShardManager - shard-manager-config53: Received role changed for member-1-shard-astronauts-config53 from Leader to Follower 04:02:13,975 PM [main] [INFO] ShardManager - shard-manager-config53 Received follower initial sync status for member-1-shard-astronauts-config53 status sync done true 04:02:13,975 PM [main] [INFO] ShardManagerTest - testWhenMultipleShardsPresentSyncStatusMustBeTrueForAllShards ending 04:02:14,76 PM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$o] 04:02:14,87 PM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config53 04:02:14,89 PM [main] [INFO] ShardManagerTest - testAddShardReplicaWithFindPrimaryTimeout starting 04:02:14,92 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config54 04:02:14,93 PM [test-akka.actor.default-dispatcher-17] [INFO] ShardManager - Recovery complete : shard-manager-config54 04:02:14,102 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:02:14,102 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config54: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://non-existent@127.0.0.1:5 04:02:14,102 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config54: All Shards are ready - data store config54 is ready 04:02:14,102 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config54: onAddShardReplica: AddShardReplica[ShardName=astronauts] 04:02:14,103 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config54: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=astronauts, waitUntilReady=true] 04:02:14,103 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config54: findPrimary for astronauts forwarding to remote ShardManager akka://non-existent@127.0.0.1:5/user/shardmanager-config54, visitedAddresses: [akka://test@127.0.0.1:2550/user/shardmanager-config54] 04:02:14,119 PM [test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://non-existent@127.0.0.1:5], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:5,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 04:02:14,119 PM [test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://non-existent@127.0.0.1:5], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:5,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 04:02:14,318 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config54: Received failure from FindPrimary for shard astronauts akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://test/user/shardmanager-config54#753426511]] after [200 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.FindPrimary]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 04:02:14,318 PM [main] [INFO] ShardManagerTest - testAddShardReplicaWithFindPrimaryTimeout ending 04:02:14,420 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/shardmanager-config54#753426511] 04:02:14,430 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config54 04:02:14,433 PM [main] [INFO] ShardManagerTest - testOnCreateShard starting 04:02:14,435 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config55 04:02:14,436 PM [test-akka.actor.default-dispatcher-17] [INFO] ShardManager - Recovery complete : shard-manager-config55 04:02:14,437 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 04:02:14,446 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config55: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=foo-ns, moduleName=foo-module, shardName=foo, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-5}, MemberName{name=member-6}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@3e851104] 04:02:14,446 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config55 doCreateShard: shardId: member-1-shard-foo-config55, memberNames: [MemberName{name=member-1}, MemberName{name=member-5}, MemberName{name=member-6}], peerAddresses: {member-6-shard-foo-config55=null, member-5-shard-foo-config55=null}, isActiveMember: true 04:02:14,447 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config55: findLocalShard : foo 04:02:14,447 PM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config55: Scheduling 300000 ms timer to wait for shard ShardInformation [shardId=member-1-shard-foo-config55, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 04:02:14,447 PM [test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-foo-config55, persistent : false 04:02:14,448 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$C/member-1-shard-foo-config55/shard-member-1:snapshot-read#-930970500] 04:02:14,448 PM [test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$C/member-1-shard-foo-config55/member-1-shard-foo-config55-notifier#-225785416 created and ready for shard:member-1-shard-foo-config55 04:02:14,449 PM [test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-foo-config55 with journal batch size 1 04:02:14,453 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-foo-config55: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 04:02:14,453 PM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-foo-config55: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 04:02:14,453 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config55: Initializing shard [foo] 04:02:14,453 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-foo-config55 is initialized 04:02:14,453 PM [test-akka.actor.default-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-foo-config55 is initialized - notifying 1 OnShardInitialized callbacks 04:02:14,453 PM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-foo-config55: onLeadershipChange, isLeader: false, hasLeader : false 04:02:14,453 PM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-foo-config55: onStateChanged: Closing all transaction chains because shard member-1-shard-foo-config55 is no longer the leader 04:02:14,453 PM [test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config55 , received role change from null to Follower 04:02:14,453 PM [test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config55 , registered listener akka://test/user/$C 04:02:14,453 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config55: Received role changed for member-1-shard-foo-config55 from null to Follower 04:02:14,455 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config55: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=foo-ns, moduleName=foo-module, shardName=foo, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-5}, MemberName{name=member-6}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@3e851104] 04:02:14,455 PM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config55: Shard foo already exists 04:02:14,455 PM [main] [INFO] ShardManagerTest - testOnCreateShard ending 04:02:14,557 PM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$C#602878013] 04:02:14,568 PM [test-shard-dispatcher-19] [INFO] Shard - Stopping Shard member-1-shard-foo-config55 04:02:14,568 PM [test-shard-dispatcher-19] [DEBUG] MessageAssembler - member-1-shard-foo-config55 (Follower): Closing 04:02:14,568 PM [test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-foo-config55: Aborting 0 pending queued transactions 04:02:14,568 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config55 04:02:14,569 PM [test-akka.actor.default-dispatcher-17] [INFO] ShardManager - Stopping ShardManager shard-manager-config1 04:02:14,569 PM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config35 04:02:14,569 PM [test-akka.actor.default-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-config43 04:02:14,569 PM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config11 04:02:14,570 PM [test-akka.actor.default-dispatcher-17] [INFO] ShardManager - Stopping ShardManager shard-manager-config21 04:02:14,570 PM [test-akka.actor.default-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-config46 04:02:14,574 PM [test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 04:02:14,574 PM [test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 04:02:14,574 PM [test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [01/16/2024 16:02:14.583] [test-akka.actor.internal-dispatcher-4] [Cluster(akka://test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config