11:59:15,45 AM [test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:59:15,65 AM [test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://test@127.0.0.1:2565] with UID [6659553090550715506] 11:59:15,76 AM [test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://test@127.0.0.1:2565] - Starting up, Akka version [2.6.21] ... 11:59:15,82 AM [test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://test@127.0.0.1:2565] - Registered cluster JMX MBean [akka:type=Cluster] 11:59:15,82 AM [test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://test@127.0.0.1:2565] - Started up successfully 11:59:15,82 AM [test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://test@127.0.0.1:2565] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:59:15,82 AM [test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://test@127.0.0.1:2565] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:59:15,142 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@182381d8 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-3-1, state=OPEN} 11:59:15,142 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-3-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1726321156]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1529/0x00000008014aca88@18e4fb75 11:59:15,142 AM [Cleaner-0] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4e80f41c transmitting entry ConnectionEntry{request=AbortLocalTransactionRequest{target=member-1-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1726321156]}, enqueuedTicks=536439830901} 11:59:15,142 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5bcd02b9 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-3-2, state=OPEN} 11:59:15,142 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-3-2, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-3-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1726321156]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1529/0x00000008014aca88@1829bcd6 11:59:15,142 AM [Cleaner-0] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2c05287f transmitting entry ConnectionEntry{request=AbortLocalTransactionRequest{target=member-1-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-3-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1726321156]}, enqueuedTicks=536439960024} 11:59:15,142 AM [Cleaner-0] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-1-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-3-0} 11:59:15,207 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config1 11:59:15,207 AM [test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$a#932375318] to Actor[akka://test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:59:15,218 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config1 11:59:15,218 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config1: Creating local shard: member-1-shard-default-config1 11:59:15,219 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:15,219 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config1 11:59:15,219 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config1: Initializing shard [default] 11:59:15,219 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config1 is initialized 11:59:15,219 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config1: Received role changed for member-1-shard-default-config1 from null to Follower 11:59:15,220 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config1: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=default, meberVotingStatusMap={member-2=true}] 11:59:15,223 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config1: findLocalShard : default 11:59:15,224 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-default-config1=true}, serversVisited=[]] to local shard akka://test/user/member-1-shard-default-config1 11:59:15,226 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config1: Received ServerChangeReply{status=NO_LEADER} from local shard akka://test/user/member-1-shard-default-config1 11:59:15,226 AM [test-akka.actor.default-dispatcher-10] [WARN] ShardManager - shard-manager-config1: ChangeServersVotingStatus failed for shard default with status NO_LEADER 11:59:15,366 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-default-config1#-1088033433] 11:59:15,380 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForShardLeader starting 11:59:15,383 AM [test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$b#1460438932] to Actor[akka://test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:59:15,383 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config2 11:59:15,385 AM [test-akka.actor.default-dispatcher-4] [INFO] ShardManager - Recovery complete : shard-manager-config2 11:59:15,385 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config2: Creating local shard: member-1-shard-default-config2 11:59:15,394 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:15,394 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config2 11:59:15,394 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 11:59:15,394 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config2: Scheduling 10000 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config2, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 11:59:15,544 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config2: Initializing shard [default] 11:59:15,544 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config2 is initialized 11:59:15,544 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config2 is initialized - notifying 1 OnShardInitialized callbacks 11:59:15,694 AM [test-akka.actor.default-dispatcher-4] [INFO] ShardManager - shard-manager-config2: Received role changed for member-1-shard-default-config2 from Candidate to Leader 11:59:15,694 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardInformation - Shard member-1-shard-default-config2 is initialized - notifying 1 OnShardInitialized callbacks 11:59:15,845 AM [test-akka.actor.default-dispatcher-4] [INFO] ShardManager - shard-manager-config2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config2, leaderId=member-1-shard-default-config2, leaderPayloadVersion=13] 11:59:15,845 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardInformation - Shard member-1-shard-default-config2 is ready - notifying 1 OnShardInitialized callbacks 11:59:15,845 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config2: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://test@127.0.0.1:2565/user/member-1-shard-default-config#1545211377, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$g5mTqNk8@7185afb1] 11:59:15,845 AM [test-akka.actor.default-dispatcher-4] [INFO] ShardManager - shard-manager-config2: All Shards are ready - data store config2 is ready 11:59:16,45 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForShardLeader ending 11:59:16,147 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$b#1460438932] 11:59:16,158 AM [test-akka.actor.default-dispatcher-4] [INFO] ShardManager - Stopping ShardManager shard-manager-config2 11:59:16,165 AM [test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://test/user/$$b] to Actor[akka://test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:59:16,165 AM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config3 11:59:16,167 AM [test-akka.actor.default-dispatcher-4] [INFO] ShardManager - Recovery complete : shard-manager-config3 11:59:16,167 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config3: Creating local shard: member-1-shard-default-config3 11:59:16,268 AM [main] [INFO] ShardManager - shard-manager-config3: Received role changed for member-1-shard-default-config3 from Follower to Candidate 11:59:16,268 AM [main] [INFO] ShardManager - shard-manager-config3 Received follower initial sync status for member-1-shard-default-config3 status sync done true 11:59:16,369 AM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$b] 11:59:16,380 AM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config3 11:59:16,393 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:59:16,403 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [3747657768972097333] 11:59:16,404 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:59:16,407 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:59:16,407 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:59:16,424 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:59:16,424 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:59:16,424 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:59:16,424 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:59:16,424 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:59:16,426 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config4] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:59:16,453 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config4 11:59:16,457 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:59:16,457 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config4 11:59:16,458 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config4: Creating local shard: member-1-shard-default-config4 11:59:16,458 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config4: Creating local shard: member-1-shard-astronauts-config4 11:59:16,458 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config4: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:59:16,458 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-default-config4 11:59:16,458 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-astronauts-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-astronauts-config4 11:59:16,474 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-6361667283464474829] 11:59:16,474 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:59:16,474 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:59:16,474 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:59:16,482 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:59:16,482 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:59:16,486 AM [main] [ERROR] ShardManagerTest - Mock Shard Leader Actor : TestActor[akka://cluster-test/user/member-2-shard-default-config4] 11:59:16,491 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config4 11:59:16,492 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config4] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:59:16,492 AM [main] [ERROR] ShardManagerTest - Forwarding actor : Actor[akka://cluster-test/user/shardmanager-config4/member-2-shard-default-config4#-1155580120] 11:59:16,507 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:16,507 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config4 11:59:16,507 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-default-config4 11:59:16,507 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-default-config4 to akka://cluster-test/user/member-1-shard-default-config 11:59:16,507 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config4: updated peer member-1-shard-default-config4 on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-default-config4 on shard member-1-shard-default-config4 whose actor address is Actor[akka://cluster-test/user/member-1-shard-default-config#400877712] 11:59:16,507 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-astronauts-config4 11:59:16,507 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-astronauts-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-astronauts-config4 11:59:16,507 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-astronauts-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-astronauts-config4 to akka://cluster-test/user/member-1-shard-default-config 11:59:16,507 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config4: updated peer member-1-shard-astronauts-config4 on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-astronauts-config4 on shard member-1-shard-astronauts-config4 whose actor address is Actor[akka://cluster-test/user/member-1-shard-default-config#400877712] 11:59:16,507 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config4: Initializing shard [default] 11:59:16,507 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config4 is initialized 11:59:16,507 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config4: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config4, leaderId=member-2-shard-default-config4, leaderPayloadVersion=12] 11:59:16,508 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config4: Received role changed for member-1-shard-default-config4 from Candidate to Follower 11:59:16,520 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config4 11:59:16,520 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config4: Creating local shard: member-2-shard-default-config4 11:59:16,521 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:16,521 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-default-config4 11:59:16,521 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config4: Initializing shard [default] 11:59:16,521 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-default-config4 is initialized 11:59:16,521 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config4: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-config4, leaderId=member-2-shard-default-config4, leaderPayloadVersion=12] 11:59:16,521 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config4: Received role changed for member-2-shard-default-config4 from Candidate to Leader 11:59:16,521 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config4: All Shards are ready - data store config4 is ready 11:59:16,877 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:59:16,931 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:59:16,931 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config4: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:59:16,931 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-default-config4 11:59:16,931 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config4 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config4/member-1-shard-default-config4 to akka://cluster-test/user/member-2-shard-default-config4 11:59:16,932 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config4: All Shards are ready - data store config4 is ready 11:59:17,431 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:59:17,435 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config4: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:59:17,435 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config4 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-default-config4 11:59:17,435 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config4 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-default-config4 to akka://cluster-test/user/member-1-shard-default-config 11:59:17,435 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-astronauts-config4 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-astronauts-config4 11:59:17,435 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-astronauts-config4 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-astronauts-config4 to akka://cluster-test/user/member-1-shard-default-config 11:59:17,438 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config4: onRemoveShardReplica: RemoveShardReplica [shardName=default, memberName=MemberName{name=member-1}] 11:59:17,442 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config4: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 11:59:17,442 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config4: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-default-config4, primaryVersion=12] 11:59:17,443 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config4: Sending RemoveServer message to peer akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-default-config4 for shard member-1-shard-default-config4 11:59:17,459 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config4: Received RemoveServerReply{status=OK, leaderHint=member-2-shard-default-config4} for shard default 11:59:17,459 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config4: Leader shard successfully removed the replica shard default 11:59:17,508 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config4: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:59:17,508 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config4 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-default-config4 11:59:17,508 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config4 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config4/member-2-shard-default-config4 to akka://cluster-test/user/member-2-shard-default-config4 11:59:17,509 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config4: All Shards are ready - data store config4 is ready 11:59:17,590 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config4 11:59:17,595 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:59:17,595 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:59:17,622 AM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:59:17,622 AM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:59:17,628 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:59:17,642 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config4 11:59:17,650 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:59:17,650 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:59:18,668 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/26/2024 11:59:18.671] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:59:18,690 AM [test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$c#1490197436] to Actor[akka://test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:59:18,691 AM [test-akka.actor.default-dispatcher-4] [INFO] ShardManager - Starting ShardManager shard-manager-config5 11:59:18,699 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config5 11:59:18,699 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config5: Creating local shard: member-1-shard-default-config5 11:59:18,706 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:18,706 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config5 11:59:18,706 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config5: Initializing shard [default] 11:59:18,706 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config5 is initialized 11:59:18,858 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$c#1490197436] 11:59:18,869 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config5 11:59:18,875 AM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config6 11:59:18,876 AM [test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://test/user/$$c] to Actor[akka://test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:59:18,877 AM [test-akka.actor.default-dispatcher-4] [INFO] ShardManager - Recovery complete : shard-manager-config6 11:59:18,877 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config6: Creating local shard: member-1-shard-default-config6 11:59:18,976 AM [main] [INFO] ShardManager - shard-manager-config6: Received role changed for member-1-shard-default-config6 from null to Follower 11:59:18,987 AM [main] [INFO] ShardManager - shard-manager-config6: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://test 11:59:18,987 AM [main] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config6 with address akka://test/user/shardmanager-config6/member-2-shard-default-config6 11:59:18,988 AM [main] [INFO] ShardManager - shard-manager-config6: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config6, leaderId=member-2-shard-default-config6, leaderPayloadVersion=13] 11:59:18,988 AM [main] [INFO] ShardManager - shard-manager-config6: All Shards are ready - data store config6 is ready 11:59:19,89 AM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$c] 11:59:19,100 AM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config6 11:59:19,108 AM [test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$d#-1876300055] to Actor[akka://test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:59:19,108 AM [test-akka.actor.default-dispatcher-4] [INFO] ShardManager - Starting ShardManager shard-manager-config7 11:59:19,109 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config7 11:59:19,119 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config7: onRemoveShardReplica: RemoveShardReplica [shardName=model-inventory, memberName=MemberName{name=member-1}] 11:59:19,120 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config7: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=model-inventory, waitUntilReady=true] 11:59:19,120 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config7: No shard found for model-inventory 11:59:19,120 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config7: Failed to find leader for shard model-inventory: received response: {} org.opendaylight.controller.cluster.datastore.exceptions.PrimaryNotFoundException: No primary shard found for model-inventory. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.findPrimary(ShardManager.java:1051) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:207) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManagerTest$TestShardManager.handleCommand(ShardManagerTest.java:2199) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:59:19,222 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$d#-1876300055] 11:59:19,233 AM [test-akka.actor.default-dispatcher-4] [INFO] ShardManager - Stopping ShardManager shard-manager-config7 11:59:19,237 AM [main] [INFO] ShardManagerTest - testAddShardReplicaWithPreExistingLocalReplicaLeader starting 11:59:19,240 AM [test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$e#-134044635] to Actor[akka://test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:59:19,241 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config8 11:59:19,243 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config8 11:59:19,243 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config8: Creating local shard: member-1-shard-default-config8 11:59:19,252 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:19,252 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config8 11:59:19,252 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config8: Initializing shard [default] 11:59:19,252 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-default-config8 is initialized 11:59:19,252 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config8: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config8, leaderId=member-1-shard-default-config8, leaderPayloadVersion=13] 11:59:19,252 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config8: Received role changed for member-1-shard-default-config8 from Candidate to Leader 11:59:19,252 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config8: All Shards are ready - data store config8 is ready 11:59:19,253 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config8: onAddShardReplica: AddShardReplica[ShardName=default] 11:59:19,254 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config8: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 11:59:19,254 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config8: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://test@127.0.0.1:2565/user/member-1-shard-default-config#107874285, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$g5mTqNk8@525e0c73] 11:59:19,254 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config8: Local shard default already exists 11:59:19,255 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config8: findLocalShard : default 11:59:19,255 AM [main] [INFO] ShardManagerTest - testAddShardReplicaWithPreExistingLocalReplicaLeader ending 11:59:19,359 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$e#-134044635] 11:59:19,370 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config8 11:59:19,373 AM [main] [INFO] ShardManagerTest - testOnReceiveFindLocalShardWaitForShardInitialized starting 11:59:19,376 AM [test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$f#19089447] to Actor[akka://test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:59:19,376 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config9 11:59:19,383 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config9 11:59:19,383 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config9: Creating local shard: member-1-shard-default-config9 11:59:19,387 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:19,387 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config9 11:59:19,387 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config9: findLocalShard : default 11:59:19,387 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config9: Scheduling 600 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config9, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 11:59:19,387 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config9: Initializing shard [default] 11:59:19,387 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardInformation - Shard member-1-shard-default-config9 is initialized 11:59:19,387 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardInformation - Shard member-1-shard-default-config9 is initialized - notifying 1 OnShardInitialized callbacks 11:59:19,392 AM [main] [INFO] ShardManagerTest - testOnReceiveFindLocalShardWaitForShardInitialized starting 11:59:19,493 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$f#19089447] 11:59:19,504 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config9 11:59:19,507 AM [main] [INFO] ShardManagerTest - testShardAvailabilityChangeOnMemberUnreachableAndLeadershipChange starting 11:59:19,516 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 11:59:19,528 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-5877308200738279019] 11:59:19,529 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:59:19,532 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:59:19,532 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:59:19,533 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:59:19,533 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:59:19,548 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:59:19,551 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config10] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:59:19,554 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config10 11:59:19,566 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:59:19,567 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:59:19,573 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config10 11:59:19,573 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config10: Creating local shard: member-1-shard-default-config10 11:59:19,574 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config10: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:59:19,574 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config10 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config10/member-1-shard-default-config10 11:59:19,576 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 11:59:19,590 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [8626397052467042368] 11:59:19,590 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:59:19,592 AM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:59:19,592 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:59:19,605 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:59:19,605 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:59:19,611 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config10] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:59:19,613 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Starting ShardManager shard-manager-config10 11:59:19,617 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:19,617 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config10 11:59:19,617 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config10 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config10/member-1-shard-default-config10 11:59:19,617 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config10 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config10/member-1-shard-default-config10 to akka://cluster-test/user/member-1-shard-default-config 11:59:19,617 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config10: updated peer member-1-shard-default-config10 on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config10/member-1-shard-default-config10 on shard member-1-shard-default-config10 whose actor address is Actor[akka://cluster-test/user/member-1-shard-default-config#895657390] 11:59:19,617 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config10: Initializing shard [default] 11:59:19,617 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-default-config10 is initialized 11:59:19,617 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config10: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config10, leaderId=member-2-shard-default-config10, leaderPayloadVersion=13] 11:59:19,618 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config10: Received role changed for member-1-shard-default-config10 from Candidate to Follower 11:59:19,643 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config10 11:59:19,643 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config10: Creating local shard: member-2-shard-default-config10 11:59:19,644 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:19,644 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-2-shard-default-config10 11:59:19,644 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config10: Initializing shard [default] 11:59:19,644 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-2-shard-default-config10 is initialized 11:59:19,644 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config10: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-config10, leaderId=member-2-shard-default-config10, leaderPayloadVersion=13] 11:59:19,644 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config10: Received role changed for member-2-shard-default-config10 from Candidate to Leader 11:59:19,644 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config10: All Shards are ready - data store config10 is ready 11:59:19,700 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:59:19,705 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:59:19,707 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config10: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:59:19,707 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config10 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config10/member-1-shard-default-config10 11:59:19,707 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config10 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config10/member-1-shard-default-config10 to akka://cluster-test/user/member-2-shard-default-config 11:59:19,707 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config10: All Shards are ready - data store config10 is ready 11:59:20,544 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:59:20,544 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config10: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:59:20,544 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config10 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config10/member-2-shard-default-config10 11:59:20,544 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config10 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config10/member-2-shard-default-config10 to akka://cluster-test/user/member-1-shard-default-config 11:59:20,545 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config10: All Shards are ready - data store config10 is ready 11:59:20,545 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config10: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 11:59:20,545 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config10: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config10/member-2-shard-default-config10, primaryVersion=13] 11:59:20,546 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 11:59:20,546 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - Marking Leader member-2-shard-default-config10 as unavailable. 11:59:20,547 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config10: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 11:59:20,547 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config10: Scheduling 1200 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config10, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=member-2-shard-default-config10, activeMember=true] 11:59:20,623 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardManager - shard-manager-config10: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:59:20,623 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config10 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config10/member-2-shard-default-config10 11:59:20,623 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config10 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config10/member-2-shard-default-config10 to akka://cluster-test/user/member-2-shard-default-config 11:59:20,623 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardManager - shard-manager-config10: All Shards are ready - data store config10 is ready 11:59:21,762 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config10: Received ShardNotInitializedTimeout message for shard default 11:59:21,762 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config10: Returning NoShardLeaderException for shard default 11:59:21,763 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardManager - shard-manager-config10: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config10, leaderId=member-1-shard-default-config10, leaderPayloadVersion=13] 11:59:21,763 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardManager - shard-manager-config10: All Shards are ready - data store config10 is ready 11:59:21,763 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardManager - shard-manager-config10: Received role changed for member-1-shard-default-config10 from Follower to Leader 11:59:21,763 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardManager - shard-manager-config10: All Shards are ready - data store config10 is ready 11:59:21,763 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config10: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 11:59:21,763 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config10: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/member-1-shard-default-config#895657390, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$g5mTqNk8@1a2cbe42] 11:59:21,763 AM [main] [INFO] ShardManagerTest - testShardAvailabilityChangeOnMemberUnreachableAndLeadershipChange ending 11:59:21,865 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-config10 11:59:21,866 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:59:21,866 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:59:21,876 AM [cluster-test-akka.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:59:21,877 AM [cluster-test-akka.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:59:21,878 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:59:21,895 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Stopping ShardManager shard-manager-config10 11:59:21,895 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:59:21,895 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:59:22,913 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/26/2024 11:59:22.916] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:59:22,936 AM [test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://test/user/shardmanager-config11] to Actor[akka://test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:59:22,936 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config11 11:59:22,938 AM [test-akka.actor.default-dispatcher-4] [INFO] ShardManager - Recovery complete : shard-manager-config11 11:59:22,958 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:22,958 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config11: onAddShardReplica: AddShardReplica[ShardName=astronauts] 11:59:22,959 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config11: Sending AddServer message to peer akka://test@127.0.0.1:2565/system/testProbe-27#-1324451259 for shard member-1-shard-astronauts-config11 11:59:22,971 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config11: onRemoveShardReplica: RemoveShardReplica [shardName=astronauts, memberName=MemberName{name=member-2}] 11:59:22,971 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config11: A shard replica operation for astronauts is already in progress 11:59:23,75 AM [main] [INFO] ShardManagerTest - testPerShardDatastoreContext starting 11:59:23,83 AM [test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://test/user/$g#937020357] to Actor[akka://test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:59:23,83 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config12 11:59:23,84 AM [test-akka.actor.default-dispatcher-4] [INFO] ShardManager - Recovery complete : shard-manager-config12 11:59:23,84 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config12: Creating local shard: member-1-shard-default-config12 11:59:23,85 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config12: Creating local shard: member-1-shard-topology-config12 11:59:23,96 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:23,96 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config12 11:59:23,96 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-topology-config12 11:59:23,97 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-default-config12 11:59:23,98 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-topology-config12 11:59:23,149 AM [main] [INFO] ShardManagerTest - testPerShardDatastoreContext ending 11:59:23,250 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/default1#1269305036] 11:59:23,261 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/topology2#889336051] 11:59:23,262 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$g#937020357] 11:59:23,262 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-config12 11:59:23,265 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForLocalLeaderShard starting 11:59:23,268 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config13 11:59:23,269 AM [test-akka.actor.default-dispatcher-4] [INFO] ShardManager - Recovery complete : shard-manager-config13 11:59:23,269 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config13: Creating local shard: member-1-shard-default-config13 11:59:23,278 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:23,278 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config13 11:59:23,279 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config13: Initializing shard [default] 11:59:23,279 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-default-config13 is initialized 11:59:23,279 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config13: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config13, leaderId=member-1-shard-default-config13, leaderPayloadVersion=13] 11:59:23,279 AM [test-akka.actor.default-dispatcher-4] [INFO] ShardManager - shard-manager-config13: Received role changed for member-1-shard-default-config13 from Candidate to Leader 11:59:23,279 AM [test-akka.actor.default-dispatcher-4] [INFO] ShardManager - shard-manager-config13: All Shards are ready - data store config13 is ready 11:59:23,279 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config13: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 11:59:23,279 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config13: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://test@127.0.0.1:2565/user/member-1-shard-default-config#-1137424215, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$g5mTqNk8@6330987e] 11:59:23,279 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForLocalLeaderShard ending 11:59:23,381 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$h#-1489961086] 11:59:23,392 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config13 11:59:23,395 AM [main] [INFO] ShardManagerTest - testShardAvailabilityChangeOnMemberWithNameContainedInLeaderIdUnreachable starting 11:59:23,404 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:59:23,425 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2562] with UID [2206208503053612679] 11:59:23,426 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Starting up, Akka version [2.6.21] ... 11:59:23,428 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Registered cluster JMX MBean [akka:type=Cluster] 11:59:23,428 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Started up successfully 11:59:23,433 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:59:23,433 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:59:23,444 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Node [akka://cluster-test@127.0.0.1:2562] is JOINING itself (with roles [member-256, dc-default], version [0.0.0]) and forming new cluster 11:59:23,451 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config14] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:59:23,451 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - is the new leader among reachable nodes (more leaders may exist) 11:59:23,451 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Leader is moving node [akka://cluster-test@127.0.0.1:2562] to [Up] 11:59:23,458 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shard-manager-config14 11:59:23,459 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:59:23,464 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config14 11:59:23,464 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config14: Creating local shard: member-256-shard-default-config14 11:59:23,465 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config14: Received MemberUp: memberName: MemberName{name=member-256}, address: akka://cluster-test@127.0.0.1:2562 11:59:23,465 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-256-shard-default-config14 with address akka://cluster-test@127.0.0.1:2562/user/shardmanager-config14/member-256-shard-default-config14 11:59:23,478 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [942452355010090698] 11:59:23,478 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:59:23,478 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:59:23,478 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:59:23,481 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:59:23,481 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:59:23,495 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config14] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:59:23,498 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config14 11:59:23,501 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:23,501 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Creating Shard member-256-shard-default-config14 11:59:23,501 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardInformation - updatePeerAddress for peer member-256-shard-default-config14 with address akka://cluster-test@127.0.0.1:2562/user/shardmanager-config14/member-256-shard-default-config14 11:59:23,501 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-256-shard-default-config14 with address akka://cluster-test@127.0.0.1:2562/user/shardmanager-config14/member-256-shard-default-config14 to akka://cluster-test/user/member-256-shard-default-config 11:59:23,501 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config14: updated peer member-256-shard-default-config14 on member MemberName{name=member-256} with address akka://cluster-test@127.0.0.1:2562/user/shardmanager-config14/member-256-shard-default-config14 on shard member-256-shard-default-config14 whose actor address is Actor[akka://cluster-test/user/member-256-shard-default-config#-1257627509] 11:59:23,501 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config14: Initializing shard [default] 11:59:23,501 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Shard member-256-shard-default-config14 is initialized 11:59:23,501 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config14: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-256-shard-default-config14, leaderId=member-256-shard-default-config14, leaderPayloadVersion=13] 11:59:23,501 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config14: Received role changed for member-256-shard-default-config14 from Candidate to Leader 11:59:23,501 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config14: All Shards are ready - data store config14 is ready 11:59:23,504 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shard-manager-config14 11:59:23,504 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config14: Creating local shard: member-2-shard-default-config14 11:59:23,504 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:23,504 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Creating Shard member-2-shard-default-config14 11:59:23,504 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config14: Initializing shard [default] 11:59:23,504 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Shard member-2-shard-default-config14 is initialized 11:59:23,504 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config14: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-config14, leaderId=member-256-shard-default-config14, leaderPayloadVersion=13] 11:59:23,505 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config14: Received role changed for member-2-shard-default-config14 from Candidate to Follower 11:59:23,547 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:59:23,554 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config14: Received MemberUp: memberName: MemberName{name=member-256}, address: akka://cluster-test@127.0.0.1:2562 11:59:23,554 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardInformation - updatePeerAddress for peer member-256-shard-default-config14 with address akka://cluster-test@127.0.0.1:2562/user/shardmanager-config14/member-256-shard-default-config14 11:59:23,554 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-256-shard-default-config14 with address akka://cluster-test@127.0.0.1:2562/user/shardmanager-config14/member-256-shard-default-config14 to akka://cluster-test/user/member-2-shard-default-config 11:59:23,554 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config14: All Shards are ready - data store config14 is ready 11:59:23,554 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2562] 11:59:24,452 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2562] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:59:24,452 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config14: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:59:24,452 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config14 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config14/member-2-shard-default-config14 11:59:24,452 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config14 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config14/member-2-shard-default-config14 to akka://cluster-test/user/member-256-shard-default-config 11:59:24,452 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config14: All Shards are ready - data store config14 is ready 11:59:24,453 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config14: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 11:59:24,453 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config14: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2562/user/member-256-shard-default-config#-1257627509, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$g5mTqNk8@1389e498] 11:59:24,454 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 11:59:24,455 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config14: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 11:59:24,455 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config14: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2562/user/member-256-shard-default-config#-1257627509, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$g5mTqNk8@1389e498] 11:59:24,456 AM [main] [INFO] ShardManagerTest - testShardAvailabilityChangeOnMemberWithNameContainedInLeaderIdUnreachable ending 11:59:24,507 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config14: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:59:24,508 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config14 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config14/member-2-shard-default-config14 11:59:24,508 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config14 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config14/member-2-shard-default-config14 to akka://cluster-test/user/member-2-shard-default-config 11:59:24,508 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config14: All Shards are ready - data store config14 is ready 11:59:24,559 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config14 11:59:24,560 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:59:24,560 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:59:24,578 AM [cluster-test-akka.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:59:24,578 AM [cluster-test-akka.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:59:24,581 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:59:24,612 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config14 11:59:24,613 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:59:24,613 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:59:25,518 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - is the new leader among reachable nodes (more leaders may exist) 11:59:25,628 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/26/2024 11:59:25.630] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:59:25,642 AM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config15 11:59:25,645 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config15 11:59:25,645 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config15: Creating local shard: member-1-shard-default-config15 11:59:25,646 AM [main] [INFO] ShardManager - shard-manager-config15: Received role changed for member-1-shard-default-config15 from Follower to Leader 11:59:25,747 AM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$d] 11:59:25,758 AM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config15 11:59:25,764 AM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config16 11:59:25,766 AM [test-akka.actor.default-dispatcher-4] [INFO] ShardManager - Recovery complete : shard-manager-config16 11:59:25,766 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config16: Creating local shard: member-1-shard-default-config16 11:59:25,766 AM [main] [INFO] ShardManager - shard-manager-config16: Received role changed for member-1-shard-default-config16 from Candidate to Leader 11:59:25,767 AM [main] [INFO] ShardManager - shard-manager-config16: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config16, leaderId=member-1-shard-default-config16, leaderPayloadVersion=13] 11:59:25,767 AM [main] [INFO] ShardManager - shard-manager-config16: All Shards are ready - data store config16 is ready 11:59:25,869 AM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$e] 11:59:25,880 AM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config16 11:59:25,882 AM [main] [INFO] ShardManagerTest - testOnCreateShardWithLocalMemberNotInShardConfig starting 11:59:25,887 AM [test-akka.actor.default-dispatcher-4] [INFO] ShardManager - Starting ShardManager shard-manager-config17 11:59:25,889 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config17 11:59:25,889 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:25,901 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config17: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=foo-ns, moduleName=foo-module, shardName=foo, shardMemberNames=[MemberName{name=member-5}, MemberName{name=member-6}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@7f78a21f] 11:59:25,901 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config17 doCreateShard: shardId: member-1-shard-foo-config17, memberNames: [MemberName{name=member-5}, MemberName{name=member-6}], peerAddresses: {}, isActiveMember: false 11:59:25,904 AM [test-shard-dispatcher-15] [INFO] Shard - Shard created : member-1-shard-foo-config17, persistent : true 11:59:25,904 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config17: findLocalShard : foo 11:59:25,904 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config17: Scheduling 60000 ms timer to wait for shard ShardInformation [shardId=member-1-shard-foo-config17, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=false] 11:59:25,905 AM [test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$i/member-1-shard-foo-config17/member-1-shard-foo-config17-notifier#733836158 created and ready for shard:member-1-shard-foo-config17 11:59:25,905 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$i/member-1-shard-foo-config17/shard-member-1:snapshot-read#798817488] 11:59:25,905 AM [test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-1-shard-foo-config17 with journal batch size 1 11:59:25,909 AM [test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-foo-config17: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 11:59:25,909 AM [test-shard-dispatcher-15] [INFO] Shard - member-1-shard-foo-config17: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:59:25,909 AM [test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-foo-config17: onLeadershipChange, isLeader: false, hasLeader : false 11:59:25,909 AM [test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-foo-config17: onStateChanged: Closing all transaction chains because shard member-1-shard-foo-config17 is no longer the leader 11:59:25,909 AM [test-shard-dispatcher-15] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:59:25,909 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config17: Initializing shard [foo] 11:59:25,909 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-foo-config17 is initialized 11:59:25,909 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-foo-config17 is initialized - notifying 1 OnShardInitialized callbacks 11:59:25,909 AM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config17 , received role change from null to Follower 11:59:25,909 AM [main] [INFO] ShardManagerTest - testOnCreateShardWithLocalMemberNotInShardConfig ending 11:59:25,916 AM [test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config17 , registered listener akka://test/user/$i 11:59:25,916 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config17: Received role changed for member-1-shard-foo-config17 from null to Follower 11:59:26,11 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$i#1096869022] 11:59:26,22 AM [test-shard-dispatcher-15] [INFO] Shard - Stopping Shard member-1-shard-foo-config17 11:59:26,22 AM [test-shard-dispatcher-15] [DEBUG] MessageAssembler - member-1-shard-foo-config17 (Follower): Closing 11:59:26,22 AM [test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-foo-config17: Aborting 0 pending queued transactions 11:59:26,23 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-config17 11:59:26,25 AM [main] [INFO] ShardManagerTest - testServerRemovedShardActorNotRunning starting 11:59:26,28 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config18 11:59:26,30 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config18 11:59:26,30 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config18: Creating local shard: member-1-shard-default-config18 11:59:26,30 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config18: Creating local shard: member-1-shard-people-config18 11:59:26,40 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config18: findLocalShard : people 11:59:26,40 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config18: Returning NotInitializedException for shard people 11:59:26,41 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config18: findLocalShard : default 11:59:26,41 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config18: Returning NotInitializedException for shard default 11:59:26,43 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config18 : Local Shard replica for shard default has been removed 11:59:26,43 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config18: persisting the shard list [people] 11:59:26,43 AM [main] [INFO] ShardManagerTest - testServerRemovedShardActorNotRunning ending 11:59:26,43 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config18 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 11:59:26,44 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config18: Successfully deleted prior snapshots 11:59:26,144 AM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$f] 11:59:26,155 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config18 11:59:26,161 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config19 11:59:26,162 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config19 11:59:26,162 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config19: Creating local shard: member-1-shard-default-config19 11:59:26,171 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:26,171 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config19 11:59:26,171 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config19: Initializing shard [default] 11:59:26,171 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-default-config19 is initialized 11:59:26,172 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config19: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 11:59:26,172 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config19: Returning NoShardLeaderException for shard default 11:59:26,273 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$j#-261516577] 11:59:26,284 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config19 11:59:26,287 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForFollowerShardWithNoInitialLeaderId starting 11:59:26,290 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config20 11:59:26,291 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config20 11:59:26,291 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config20: Creating local shard: member-1-shard-default-config20 11:59:26,300 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:26,300 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config20 11:59:26,300 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config20: Initializing shard [default] 11:59:26,300 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-default-config20 is initialized 11:59:26,300 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config20: Received role changed for member-1-shard-default-config20 from Candidate to Follower 11:59:26,300 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config20: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 11:59:26,300 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config20: Returning NoShardLeaderException for shard default 11:59:26,302 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config20: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config20, leaderId=member-1-shard-default-config20, leaderPayloadVersion=13] 11:59:26,302 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config20: All Shards are ready - data store config20 is ready 11:59:26,302 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config20: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 11:59:26,302 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config20: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://test@127.0.0.1:2565/user/member-1-shard-default-config#-344479947, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$g5mTqNk8@51d4ac0a] 11:59:26,302 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForFollowerShardWithNoInitialLeaderId starting 11:59:26,404 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$k#996446321] 11:59:26,415 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config20 11:59:26,420 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config21 11:59:26,422 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config21 11:59:26,441 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:26,441 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config21: onAddShardReplica: AddShardReplica[ShardName=astronauts] 11:59:26,442 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config21: Sending AddServer message to peer akka://test@127.0.0.1:2565/system/testProbe-45#-119640005 for shard member-1-shard-astronauts-config21 11:59:26,454 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config21: onAddShardReplica: AddShardReplica[ShardName=astronauts] 11:59:26,455 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config21: A shard replica operation for astronauts is already in progress 11:59:26,559 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithCandidateShard starting 11:59:26,562 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config22 11:59:26,568 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config22 11:59:26,568 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config22: Creating local shard: member-1-shard-default-config22 11:59:26,572 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:26,572 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config22 11:59:26,572 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config22: Initializing shard [default] 11:59:26,572 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config22 is initialized 11:59:26,572 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config22: Received role changed for member-1-shard-default-config22 from null to Candidate 11:59:26,572 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config22: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 11:59:26,572 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config22: Scheduling 1200 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config22, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Candidate, leaderId=null, activeMember=true] 11:59:27,784 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config22: Received ShardNotInitializedTimeout message for shard default 11:59:27,784 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config22: Returning NoShardLeaderException for shard default 11:59:27,785 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithCandidateShard ending 11:59:27,886 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$l#-2096238429] 11:59:27,897 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config22 11:59:27,900 AM [main] [INFO] ShardManagerTest - testAddShardReplicaWithAddServerReplyFailure starting 11:59:27,904 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config23 11:59:27,905 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config23 11:59:28,25 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:28,35 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config23: onAddShardReplica: AddShardReplica[ShardName=astronauts] 11:59:28,36 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config23: Sending AddServer message to peer akka://test@127.0.0.1:2565/system/testProbe-50#-1053589393 for shard member-1-shard-astronauts-config23 11:59:28,38 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config23: Received AddServerReply{status=TIMEOUT} for shard astronauts from leader akka://test@127.0.0.1:2565/system/testProbe-50#-1053589393 11:59:28,38 AM [test-akka.actor.default-dispatcher-13] [WARN] ShardManager - shard-manager-config23: Leader failed to add shard replica astronauts with status TIMEOUT 11:59:28,38 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config23: findLocalShard : astronauts 11:59:28,38 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config23: Local shard astronauts not found - shards present: [] 11:59:28,38 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config23: onAddShardReplica: AddShardReplica[ShardName=astronauts] 11:59:28,39 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config23: Sending AddServer message to peer akka://test@127.0.0.1:2565/system/testProbe-50#-1053589393 for shard member-1-shard-astronauts-config23 11:59:28,40 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config23: Received AddServerReply{status=NO_LEADER} for shard astronauts from leader akka://test@127.0.0.1:2565/system/testProbe-50#-1053589393 11:59:28,40 AM [test-akka.actor.default-dispatcher-10] [WARN] ShardManager - shard-manager-config23: Leader failed to add shard replica astronauts with status NO_LEADER 11:59:28,40 AM [main] [INFO] ShardManagerTest - testAddShardReplicaWithAddServerReplyFailure ending 11:59:28,142 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-astronauts-config#1799890533] 11:59:28,153 AM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/shardmanager-config23] 11:59:28,154 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config23 11:59:28,159 AM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config24 11:59:28,161 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config24 11:59:28,161 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config24: Creating local shard: member-1-shard-default-config24 11:59:28,260 AM [main] [INFO] ShardManager - shard-manager-config24: Received role changed for member-1-shard-default-config24 from Candidate to Follower 11:59:28,261 AM [main] [INFO] ShardManager - shard-manager-config24 Received follower initial sync status for member-1-shard-default-config24 status sync done true 11:59:28,261 AM [main] [INFO] ShardManager - shard-manager-config24 Received follower initial sync status for member-1-shard-default-config24 status sync done false 11:59:28,362 AM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$g] 11:59:28,373 AM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config24 11:59:28,378 AM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config25 11:59:28,380 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config25 11:59:28,380 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config25: Creating local shard: member-1-shard-default-config25 11:59:28,380 AM [main] [INFO] ShardManager - shard-manager-config25: Received role changed for unknown from Candidate to Leader 11:59:28,482 AM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$h] 11:59:28,493 AM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config25 11:59:28,495 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithUninitializedShard starting 11:59:28,498 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config26 11:59:28,502 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config26 11:59:28,502 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config26: Creating local shard: member-1-shard-default-config26 11:59:28,509 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:28,509 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config26 11:59:28,509 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config26: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 11:59:28,509 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config26: Scheduling 600 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config26, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 11:59:29,125 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config26: Received ShardNotInitializedTimeout message for shard default 11:59:29,125 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config26: Returning NotInitializedException for shard default 11:59:29,127 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config26: Initializing shard [default] 11:59:29,127 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config26 is initialized 11:59:29,326 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithUninitializedShard ending 11:59:29,428 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$m#-1599348479] 11:59:29,439 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config26 11:59:29,444 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config27 11:59:29,446 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config27 11:59:29,446 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config27: Creating local shard: member-1-shard-default-config27 11:59:29,455 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:29,455 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config27 11:59:29,455 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config27: Initializing shard [default] 11:59:29,455 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config27 is initialized 11:59:29,455 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config27: findLocalShard : default 11:59:29,557 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$n#-363004793] 11:59:29,567 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config27 11:59:29,569 AM [main] [INFO] ShardManagerTest - testRegisterForShardLeaderChanges starting 11:59:29,571 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config28 11:59:29,573 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config28 11:59:29,573 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config28: Creating local shard: member-1-shard-default-config28 11:59:29,584 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:29,584 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config28 11:59:29,584 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config28: Initializing shard [default] 11:59:29,584 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config28 is initialized 11:59:29,661 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config28: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3af04579 11:59:29,661 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config28: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config28, leaderId=member-1-shard-default-config28, leaderPayloadVersion=13] 11:59:29,674 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config28: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config28, leaderId=member-1-shard-default-config28, leaderPayloadVersion=13] 11:59:30,175 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config28: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config28, leaderId=null, leaderPayloadVersion=13] 11:59:30,186 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config28: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config28, leaderId=member-2-shard-default-config28, leaderPayloadVersion=13] 11:59:30,197 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config28: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config28, leaderId=member-1-shard-default-config28, leaderPayloadVersion=13] 11:59:30,697 AM [main] [INFO] ShardManagerTest - testRegisterForShardLeaderChanges ending 11:59:30,799 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$o#-1258830474] 11:59:30,810 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-config28 11:59:30,814 AM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config29 11:59:30,816 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config29 11:59:30,816 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config29: Creating local shard: member-1-shard-default-config29 11:59:31,17 AM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$i] 11:59:31,28 AM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config29 11:59:31,32 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config30 11:59:31,34 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config30 11:59:31,34 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config30: Creating local shard: member-1-shard-default-config30 11:59:31,43 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:31,43 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config30 11:59:31,43 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config30: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=non-existent, waitUntilReady=false] 11:59:31,43 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config30: No shard found for non-existent 11:59:31,144 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$p#-794062298] 11:59:31,155 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config30 11:59:31,165 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config31 11:59:31,166 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config31 11:59:31,176 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config31: onAddShardReplica: AddShardReplica[ShardName=model-inventory] 11:59:31,176 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config31: No module configuration exists for shard model-inventory 11:59:31,278 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$q#1509368096] 11:59:31,289 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config31 11:59:31,292 AM [main] [INFO] ShardManagerTest - testAddShardReplicaWithPreExistingReplicaInRemoteShardLeader starting 11:59:31,295 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config32 11:59:31,297 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config32 11:59:31,297 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config32: Creating local shard: member-1-shard-default-config32 11:59:31,405 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:31,405 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config32 11:59:31,405 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config32: Initializing shard [default] 11:59:31,405 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config32 is initialized 11:59:31,405 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config32: Received MemberUp: memberName: MemberName{name=leader-member}, address: akka://test 11:59:31,405 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer leader-member-shard-default-config32 with address akka://test/user/shardmanager-config32/leader-member-shard-default-config32 11:59:31,405 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer leader-member-shard-default-config32 with address akka://test/user/shardmanager-config32/leader-member-shard-default-config32 to akka://test/user/member-1-shard-default-config 11:59:31,405 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config32: Received role changed for member-1-shard-default-config32 from Candidate to Follower 11:59:31,405 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config32: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config32, leaderId=leader-member-shard-default-config32, leaderPayloadVersion=13] 11:59:31,405 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config32: All Shards are ready - data store config32 is ready 11:59:31,405 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config32: onAddShardReplica: AddShardReplica[ShardName=default] 11:59:31,406 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config32: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 11:59:31,406 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config32: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://test/user/shardmanager-config32/leader-member-shard-default-config32, primaryVersion=13] 11:59:31,406 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config32: Sending AddServer message to peer akka://test/user/shardmanager-config32/leader-member-shard-default-config32 for shard member-1-shard-default-config32 11:59:31,406 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config32: Received AddServerReply{status=ALREADY_EXISTS} for shard default from leader akka://test/user/shardmanager-config32/leader-member-shard-default-config32 11:59:31,406 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config32: Local shard default already exists 11:59:31,456 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config32: findLocalShard : default 11:59:31,457 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config32: onAddShardReplica: AddShardReplica[ShardName=default] 11:59:31,457 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config32: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 11:59:31,457 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config32: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://test/user/shardmanager-config32/leader-member-shard-default-config32, primaryVersion=13] 11:59:31,457 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config32: Sending AddServer message to peer akka://test/user/shardmanager-config32/leader-member-shard-default-config32 for shard member-1-shard-default-config32 11:59:31,458 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config32: Received AddServerReply{status=ALREADY_EXISTS} for shard default from leader akka://test/user/shardmanager-config32/leader-member-shard-default-config32 11:59:31,458 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config32: Local shard default already exists 11:59:31,459 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-default-config32 11:59:31,459 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config32: onAddShardReplica: AddShardReplica[ShardName=default] 11:59:31,459 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config32: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 11:59:31,459 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config32: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://test/user/shardmanager-config32/leader-member-shard-default-config32, primaryVersion=13] 11:59:31,459 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config32: Sending AddServer message to peer akka://test/user/shardmanager-config32/leader-member-shard-default-config32 for shard member-1-shard-default-config32 11:59:31,575 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config32: AddServer request to akka://test/user/shardmanager-config32/leader-member-shard-default-config32 for default failed akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://test/), Path(/user/shardmanager-config32/leader-member-shard-default-config32)]] after [100 ms]. Message of type [org.opendaylight.controller.cluster.raft.messages.AddServer]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 11:59:31,576 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config32: findLocalShard : default 11:59:31,576 AM [main] [INFO] ShardManagerTest - testAddShardReplicaWithPreExistingReplicaInRemoteShardLeader ending 11:59:31,677 AM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/shardmanager-config32] 11:59:31,688 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-config32 11:59:31,690 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForRemoteShard starting 11:59:31,699 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 11:59:31,730 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-6693623624564037763] 11:59:31,730 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:59:31,735 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:59:31,735 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:59:31,741 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:59:31,741 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:59:31,753 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:59:31,753 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:59:31,753 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:59:31,753 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config33] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:59:31,759 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Starting ShardManager shard-manager-config33 11:59:31,760 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:59:31,792 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config33 11:59:31,792 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config33: Creating local shard: member-1-shard-default-config33 11:59:31,792 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config33: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:59:31,792 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-default-config33 11:59:31,810 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-6291771227281337787] 11:59:31,810 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:59:31,810 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:59:31,810 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:59:31,810 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:59:31,810 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:59:31,834 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:31,834 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config33 11:59:31,834 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-default-config33 11:59:31,834 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-default-config33 to akka://test/user/member-1-shard-default-config 11:59:31,834 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config33: updated peer member-1-shard-default-config33 on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-default-config33 on shard member-1-shard-default-config33 whose actor address is Actor[akka://test/user/member-1-shard-default-config#-927597014] 11:59:31,835 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config33] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:59:31,840 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config33 11:59:31,877 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - Recovery complete : shard-manager-config33 11:59:31,877 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config33: Creating local shard: member-2-shard-default-config33 11:59:31,878 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config33: Creating local shard: member-2-shard-astronauts-config33 11:59:31,879 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:31,879 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-2-shard-default-config33 11:59:31,879 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-2-shard-astronauts-config33 11:59:31,879 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config33: Initializing shard [astronauts] 11:59:31,879 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardInformation - Shard member-2-shard-astronauts-config33 is initialized 11:59:31,879 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config33: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-astronauts-config33, leaderId=member-2-shard-astronauts-config33, leaderPayloadVersion=12] 11:59:31,879 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config33: Received role changed for member-2-shard-astronauts-config33 from Candidate to Leader 11:59:31,917 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:59:31,918 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:59:31,918 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shard-manager-config33: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:59:31,918 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-default-config33 11:59:31,918 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-default-config33 to akka://cluster-test/user/member-2-shard-astronauts-config 11:59:31,918 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-astronauts-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-astronauts-config33 11:59:31,918 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-astronauts-config33 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33/member-1-shard-astronauts-config33 to akka://cluster-test/user/member-2-shard-astronauts-config 11:59:32,758 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:59:32,758 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config33: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:59:32,758 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config33 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33/member-2-shard-default-config33 11:59:32,758 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config33 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33/member-2-shard-default-config33 to akka://test/user/member-1-shard-default-config 11:59:32,759 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config33: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=astronauts, waitUntilReady=false] 11:59:32,759 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config33: findPrimary for astronauts forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33, visitedAddresses: [akka://cluster-test@127.0.0.1:2558/user/shardmanager-config33] 11:59:32,766 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config33: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=astronauts, waitUntilReady=false] 11:59:32,766 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config33: Found primary for astronauts: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/member-2-shard-astronauts-config#1434313425, primaryVersion=12] 11:59:32,769 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForRemoteShard ending 11:59:32,830 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardManager - shard-manager-config33: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:59:32,830 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config33 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33/member-2-shard-default-config33 11:59:32,830 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config33 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33/member-2-shard-default-config33 to akka://cluster-test/user/member-2-shard-astronauts-config 11:59:32,830 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-astronauts-config33 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33/member-2-shard-astronauts-config33 11:59:32,830 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-astronauts-config33 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config33/member-2-shard-astronauts-config33 to akka://cluster-test/user/member-2-shard-astronauts-config 11:59:32,871 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Stopping ShardManager shard-manager-config33 11:59:32,872 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:59:32,872 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:59:32,884 AM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:59:32,884 AM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:59:32,886 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:59:32,907 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Stopping ShardManager shard-manager-config33 11:59:32,908 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:59:32,908 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:59:33,923 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/26/2024 11:59:33.925] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:59:33,952 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config34 11:59:33,957 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config34 11:59:33,957 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config34: Creating local shard: member-1-shard-default-config34 11:59:33,963 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config34: findLocalShard : default 11:59:33,963 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config34: Returning NotInitializedException for shard default 11:59:34,65 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$r#1144694138] 11:59:34,76 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config34 11:59:34,83 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config35 11:59:34,84 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config35 11:59:34,84 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config35: Creating local shard: member-1-shard-default-config35 11:59:34,92 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:34,92 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config35 11:59:34,92 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config35: Initializing shard [default] 11:59:34,92 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-default-config35 is initialized 11:59:34,92 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config35: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config35, leaderId=member-1-shard-default-config35, leaderPayloadVersion=13] 11:59:34,93 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config35: Received role changed for member-1-shard-default-config35 from Candidate to Leader 11:59:34,93 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config35: All Shards are ready - data store config35 is ready 11:59:34,93 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config35: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=default, meberVotingStatusMap={member-2=true}] 11:59:34,93 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config35: findLocalShard : default 11:59:34,93 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config35: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-default-config35=true}, serversVisited=[]] to local shard akka://test/user/member-1-shard-default-config35 11:59:34,94 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config35: Received ServerChangeReply{status=OK} from local shard akka://test/user/member-1-shard-default-config35 11:59:34,94 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config35: ChangeServersVotingStatus succeeded for shard default 11:59:34,246 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-default-config35#-996482088] 11:59:34,259 AM [main] [INFO] ShardManagerTest - testShutDown starting 11:59:34,263 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config36 11:59:34,264 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config36 11:59:34,264 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config36: Creating local shard: member-1-shard-shard2-config36 11:59:34,264 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config36: Creating local shard: member-1-shard-shard1-config36 11:59:34,273 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:34,273 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-shard2-config36 11:59:34,273 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-shard1-config36 11:59:34,274 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config36: Initializing shard [shard1] 11:59:34,274 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-shard1-config36 is initialized 11:59:34,274 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config36: Initializing shard [shard2] 11:59:34,274 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-shard2-config36 is initialized 11:59:34,274 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config36: Issuing gracefulStop to shard member-1-shard-shard2-config36 11:59:34,274 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config36: Issuing gracefulStop to shard member-1-shard-shard1-config36 11:59:34,274 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-config36 - waiting on 2 shards 11:59:34,824 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-shard1-config36#1404209358] 11:59:34,825 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-shard2-config36#-1794546961] 11:59:34,826 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config36: All shards shutdown - sending PoisonPill to self 11:59:34,826 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config36 11:59:34,826 AM [main] [INFO] ShardManagerTest - testShutDown ending 11:59:34,928 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$t#-342102826] 11:59:34,941 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForNonLocalLeaderShardBeforeMemberUp starting 11:59:34,944 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config37 11:59:34,945 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config37 11:59:34,945 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config37: Creating local shard: member-1-shard-default-config37 11:59:34,954 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:34,954 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config37 11:59:34,954 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config37: Initializing shard [default] 11:59:34,954 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config37 is initialized 11:59:34,954 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config37: Received role changed for member-1-shard-default-config37 from Candidate to Follower 11:59:34,954 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManagerTest$TestShardManager - Received unhandled message LeaderStateChanged [memberId=member-1-shard-default-config37, leaderId=member-2-shard-default-config37, leaderPayloadVersion=13] 11:59:34,955 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config37: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 11:59:34,955 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config37: Returning NoShardLeaderException for shard default 11:59:34,956 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForNonLocalLeaderShardBeforeMemberUp ending 11:59:35,57 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$u#2142295454] 11:59:35,68 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config37 11:59:35,71 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithNoRoleShard starting 11:59:35,75 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config38 11:59:35,77 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config38 11:59:35,77 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config38: Creating local shard: member-1-shard-default-config38 11:59:35,86 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:35,86 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config38 11:59:35,86 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config38: Initializing shard [default] 11:59:35,86 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config38 is initialized 11:59:35,86 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config38: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 11:59:35,86 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config38: Scheduling 1200 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config38, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 11:59:36,304 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config38: Received ShardNotInitializedTimeout message for shard default 11:59:36,304 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config38: Returning NoShardLeaderException for shard default 11:59:36,305 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithNoRoleShard ending 11:59:36,407 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$v#-1687497525] 11:59:36,418 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config38 11:59:36,421 AM [main] [INFO] ShardManagerTest - testShardPersistenceWithRestoredData starting 11:59:36,425 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config39 11:59:36,426 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config39: onSnapshotOffer: ShardManagerSnapshot [ShardList = [default, astronauts] ] 11:59:36,426 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config39: adding shard astronauts 11:59:36,427 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config39: removing shard people 11:59:36,427 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config39 11:59:36,427 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config39: Creating local shard: member-1-shard-default-config39 11:59:36,427 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config39: Creating local shard: member-1-shard-astronauts-config39 11:59:36,438 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config39: findLocalShard : people 11:59:36,438 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config39: Local shard people not found - shards present: [default, astronauts] 11:59:36,438 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config39: findLocalShard : default 11:59:36,438 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config39: Returning NotInitializedException for shard default 11:59:36,438 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config39: findLocalShard : astronauts 11:59:36,438 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config39: Returning NotInitializedException for shard astronauts 11:59:36,438 AM [main] [INFO] ShardManagerTest - testShardPersistenceWithRestoredData ending 11:59:36,539 AM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$j] 11:59:36,550 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-config39 11:59:36,552 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForNonLocalLeaderShard starting 11:59:36,555 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config40 11:59:36,557 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config40 11:59:36,557 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config40: Creating local shard: member-1-shard-default-config40 11:59:36,566 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:36,566 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config40 11:59:36,566 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config40: Initializing shard [default] 11:59:36,566 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-default-config40 is initialized 11:59:36,566 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config40: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://test 11:59:36,566 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config40 with address akka://test/user/shardmanager-config40/member-2-shard-default-config40 11:59:36,566 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config40 with address akka://test/user/shardmanager-config40/member-2-shard-default-config40 to akka://test/user/member-1-shard-default-config 11:59:36,566 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config40: Received role changed for member-1-shard-default-config40 from Candidate to Follower 11:59:36,566 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config40: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config40, leaderId=member-2-shard-default-config40, leaderPayloadVersion=12] 11:59:36,567 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config40: All Shards are ready - data store config40 is ready 11:59:36,567 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config40: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 11:59:36,567 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config40: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://test/user/shardmanager-config40/member-2-shard-default-config40, primaryVersion=12] 11:59:36,567 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryForNonLocalLeaderShard ending 11:59:36,668 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$w#-501884895] 11:59:36,679 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-config40 11:59:36,681 AM [main] [INFO] ShardManagerTest - testShardAvailabilityOnChangeOfMemberReachability starting 11:59:36,692 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 11:59:36,701 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [3870224864133662178] 11:59:36,702 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:59:36,704 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:59:36,704 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:59:36,711 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:59:36,711 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:59:36,716 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:59:36,716 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:59:36,716 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:59:36,722 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config41] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:59:36,727 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Starting ShardManager shard-manager-config41 11:59:36,748 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config41 11:59:36,748 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config41: Creating local shard: member-1-shard-default-config41 11:59:36,749 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config41: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:59:36,749 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-1-shard-default-config41 11:59:36,752 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:59:36,765 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [8979057718498410545] 11:59:36,765 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:59:36,773 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:59:36,773 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:59:36,773 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:59:36,773 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:59:36,792 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config41 11:59:36,792 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config41] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:59:36,795 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:36,795 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config41 11:59:36,795 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-1-shard-default-config41 11:59:36,795 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-1-shard-default-config41 to akka://cluster-test/user/member-1-shard-default-config 11:59:36,795 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config41: updated peer member-1-shard-default-config41 on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-1-shard-default-config41 on shard member-1-shard-default-config41 whose actor address is Actor[akka://cluster-test/user/member-1-shard-default-config#1211534897] 11:59:36,795 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config41: Initializing shard [default] 11:59:36,795 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-default-config41 is initialized 11:59:36,795 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config41: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config41, leaderId=member-2-shard-default-config41, leaderPayloadVersion=13] 11:59:36,795 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config41: Received role changed for member-1-shard-default-config41 from Candidate to Follower 11:59:36,810 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config41 11:59:36,810 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config41: Creating local shard: member-2-shard-default-config41 11:59:36,811 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:36,811 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-default-config41 11:59:36,811 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config41: Initializing shard [default] 11:59:36,811 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-default-config41 is initialized 11:59:36,811 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config41: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-config41, leaderId=member-2-shard-default-config41, leaderPayloadVersion=13] 11:59:36,811 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config41: Received role changed for member-2-shard-default-config41 from Candidate to Leader 11:59:36,811 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config41: All Shards are ready - data store config41 is ready 11:59:36,847 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:59:36,853 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config41: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:59:36,853 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-1-shard-default-config41 11:59:36,853 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-1-shard-default-config41 to akka://cluster-test/user/member-2-shard-default-config 11:59:36,853 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config41: All Shards are ready - data store config41 is ready 11:59:36,853 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:59:37,728 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:59:37,729 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config41: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:59:37,729 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config41/member-2-shard-default-config41 11:59:37,729 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config41/member-2-shard-default-config41 to akka://cluster-test/user/member-1-shard-default-config 11:59:37,729 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ShardManager - shard-manager-config41: All Shards are ready - data store config41 is ready 11:59:37,729 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config41: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 11:59:37,729 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config41: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config41/member-2-shard-default-config41, primaryVersion=13] 11:59:37,731 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 11:59:37,731 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Marking Leader member-2-shard-default-config41 as unavailable. 11:59:37,731 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config41: Received MemberRemoved: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 11:59:37,731 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config41: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 11:59:37,731 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config41: Scheduling 1200 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config41, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=member-2-shard-default-config41, activeMember=true] 11:59:37,797 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config41: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:59:37,797 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config41/member-2-shard-default-config41 11:59:37,797 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config41/member-2-shard-default-config41 to akka://cluster-test/user/member-2-shard-default-config 11:59:37,797 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shard-manager-config41: All Shards are ready - data store config41 is ready 11:59:38,945 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config41: Received ShardNotInitializedTimeout message for shard default 11:59:38,946 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config41: Returning NoShardLeaderException for shard default 11:59:38,946 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Received ReachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 11:59:38,946 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41 11:59:38,946 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41 to akka://cluster-test/user/member-1-shard-default-config 11:59:38,946 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Marking Leader member-2-shard-default-config41 as available. 11:59:38,947 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config41: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 11:59:38,947 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config41: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41, primaryVersion=13] 11:59:38,947 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config41: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 11:59:38,947 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41 11:59:38,947 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41 to akka://cluster-test/user/member-1-shard-default-config 11:59:38,947 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - shard-manager-config41: All Shards are ready - data store config41 is ready 11:59:38,947 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 11:59:38,947 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Marking Leader member-2-shard-default-config41 as unavailable. 11:59:38,947 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config41: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 11:59:38,947 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config41: Scheduling 1200 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config41, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=member-2-shard-default-config41, activeMember=true] 11:59:38,947 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Received ReachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2558 11:59:38,947 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41 11:59:38,947 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config41 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41 to akka://cluster-test/user/member-1-shard-default-config 11:59:38,947 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-default-config41 is initialized - notifying 1 OnShardInitialized callbacks 11:59:38,947 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Marking Leader member-2-shard-default-config41 as available. 11:59:38,947 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-default-config41 is ready - notifying 1 OnShardInitialized callbacks 11:59:38,947 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config41: Found primary for default: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config41/member-2-shard-default-config41, primaryVersion=13] 11:59:38,948 AM [main] [INFO] ShardManagerTest - testShardAvailabilityOnChangeOfMemberReachability ending 11:59:39,49 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config41 11:59:39,50 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:59:39,50 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:59:39,59 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:59:39,60 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:59:39,62 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:59:39,78 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config41 11:59:39,80 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:59:39,80 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/26/2024 11:59:40.111] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:59:40,120 AM [main] [INFO] ShardManagerTest - testAddShardReplica starting 11:59:40,127 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 11:59:40,147 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [4761842734698829741] 11:59:40,147 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:59:40,147 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:59:40,147 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:59:40,147 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:59:40,147 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:59:40,158 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:59:40,158 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:59:40,158 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:59:40,160 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config42] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:59:40,162 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - Starting ShardManager shardmanager-config42 11:59:40,172 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:59:40,188 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-5203974556068064794] 11:59:40,189 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:59:40,190 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardManager - Recovery complete : shardmanager-config42 11:59:40,190 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shardmanager-config42: Creating local shard: member-1-shard-default-config42 11:59:40,191 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardManager - shardmanager-config42: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:59:40,191 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-default-config42 11:59:40,191 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:59:40,191 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:59:40,193 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:59:40,193 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:59:40,209 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Starting ShardManager shardmanager-config42 11:59:40,209 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[akka://cluster-test/user/shardmanager-config42] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:59:40,217 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:40,217 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config42 11:59:40,217 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-default-config42 11:59:40,217 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-default-config42 to akka://cluster-test/user/member-1-shard-default-config 11:59:40,217 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shardmanager-config42: updated peer member-1-shard-default-config42 on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-default-config42 on shard member-1-shard-default-config42 whose actor address is Actor[akka://cluster-test/user/member-1-shard-default-config#-1721661041] 11:59:40,224 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - Recovery complete : shardmanager-config42 11:59:40,224 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shardmanager-config42: Creating local shard: member-2-shard-default-config42 11:59:40,225 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shardmanager-config42: Creating local shard: member-2-shard-astronauts-config42 11:59:40,251 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:40,251 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Creating Shard member-2-shard-default-config42 11:59:40,251 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - Creating Shard member-2-shard-astronauts-config42 11:59:40,251 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shardmanager-config42: Initializing shard [astronauts] 11:59:40,251 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Shard member-2-shard-astronauts-config42 is initialized 11:59:40,251 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shardmanager-config42: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-astronauts-config42, leaderId=member-2-shard-astronauts-config42, leaderPayloadVersion=12] 11:59:40,251 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shardmanager-config42: Received role changed for member-2-shard-astronauts-config42 from Candidate to Leader 11:59:40,274 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardManager - shardmanager-config42: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:59:40,274 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-default-config42 11:59:40,274 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-default-config42 to akka://cluster-test/user/member-2-shard-astronauts-config 11:59:40,274 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-astronauts-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-astronauts-config42 11:59:40,274 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-astronauts-config42 with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42/member-1-shard-astronauts-config42 to akka://cluster-test/user/member-2-shard-astronauts-config 11:59:40,274 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:59:40,277 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:59:41,166 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:59:41,168 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shardmanager-config42: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:59:41,168 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config42 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42/member-2-shard-default-config42 11:59:41,168 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config42 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42/member-2-shard-default-config42 to akka://cluster-test/user/member-1-shard-default-config 11:59:41,171 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shardmanager-config42: onAddShardReplica: AddShardReplica[ShardName=astronauts] 11:59:41,171 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shardmanager-config42: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=astronauts, waitUntilReady=true] 11:59:41,171 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shardmanager-config42: findPrimary for astronauts forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42, visitedAddresses: [akka://cluster-test@127.0.0.1:2558/user/shardmanager-config42] 11:59:41,175 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shardmanager-config42: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=astronauts, waitUntilReady=true] 11:59:41,175 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shardmanager-config42: Found primary for astronauts: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/member-2-shard-astronauts-config, primaryVersion=12] 11:59:41,178 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shardmanager-config42: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2559/user/member-2-shard-astronauts-config for shard member-1-shard-astronauts-config42 11:59:41,184 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shardmanager-config42: Received AddServerReply{status=OK, leaderHint=member-2-shard-astronauts-config42} for shard astronauts from leader akka://cluster-test@127.0.0.1:2559/user/member-2-shard-astronauts-config 11:59:41,184 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shardmanager-config42: Leader shard successfully added the replica shard astronauts 11:59:41,184 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-astronauts-config42 11:59:41,184 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shardmanager-config42: persisting the shard list [default, astronauts] 11:59:41,185 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shardmanager-config42 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 11:59:41,186 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shardmanager-config42: Successfully deleted prior snapshots 11:59:41,218 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardManager - shardmanager-config42: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:59:41,218 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config42 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42/member-2-shard-default-config42 11:59:41,218 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-config42 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42/member-2-shard-default-config42 to akka://cluster-test/user/member-2-shard-astronauts-config 11:59:41,218 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-astronauts-config42 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42/member-2-shard-astronauts-config42 11:59:41,218 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-astronauts-config42 with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config42/member-2-shard-astronauts-config42 to akka://cluster-test/user/member-2-shard-astronauts-config 11:59:41,224 AM [main] [INFO] ShardManagerTest - testAddShardReplica ending 11:59:41,327 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shardmanager-config42 11:59:41,328 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:59:41,328 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:59:41,340 AM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:59:41,340 AM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:59:41,341 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:59:41,355 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shardmanager-config42 11:59:41,356 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:59:41,356 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:59:42,373 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/26/2024 11:59:42.375] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:59:42,396 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config43 11:59:42,396 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config43 11:59:42,397 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config43: Creating local shard: member-1-shard-default-config43 11:59:42,404 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:42,404 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config43 11:59:42,404 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config43: Initializing shard [default] 11:59:42,404 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-config43 is initialized 11:59:42,404 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config43: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config43, leaderId=member-1-shard-default-config43, leaderPayloadVersion=13] 11:59:42,404 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config43: Received role changed for member-1-shard-default-config43 from Candidate to Leader 11:59:42,404 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config43: All Shards are ready - data store config43 is ready 11:59:42,404 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config43: onRemoveShardReplica: RemoveShardReplica [shardName=default, memberName=MemberName{name=member-1}] 11:59:42,405 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config43: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 11:59:42,405 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config43: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://test@127.0.0.1:2565/user/member-1-shard-default-config43#-1389139240, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.api.DataTree$MockitoMock$g5mTqNk8@4da18e2c] 11:59:42,405 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config43: Sending RemoveServer message to peer akka://test@127.0.0.1:2565/user/member-1-shard-default-config43#-1389139240 for shard member-1-shard-default-config43 11:59:42,405 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config43: Received RemoveServerReply{status=OK} for shard default 11:59:42,405 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config43: Leader shard successfully removed the replica shard default 11:59:42,556 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-default-config43#-1389139240] 11:59:42,569 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithIsolatedLeaderShard starting 11:59:42,573 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config44 11:59:42,574 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config44 11:59:42,574 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config44: Creating local shard: member-1-shard-default-config44 11:59:42,583 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:42,583 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config44 11:59:42,583 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config44: Initializing shard [default] 11:59:42,583 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-default-config44 is initialized 11:59:42,583 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - shard-manager-config44: Received role changed for member-1-shard-default-config44 from null to IsolatedLeader 11:59:42,583 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config44: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 11:59:42,584 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config44: Scheduling 1200 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-config44, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=null, activeMember=true] 11:59:43,794 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config44: Received ShardNotInitializedTimeout message for shard default 11:59:43,794 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config44: Returning NoShardLeaderException for shard default 11:59:43,795 AM [main] [INFO] ShardManagerTest - testOnReceiveFindPrimaryWaitForReadyWithIsolatedLeaderShard ending 11:59:43,896 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$y#485074110] 11:59:43,907 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config44 11:59:43,913 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config45 11:59:43,914 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config45 11:59:43,914 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config45: Creating local shard: member-1-shard-default-config45 11:59:43,923 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:43,923 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config45 11:59:43,923 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config45: findLocalShard : non-existent 11:59:43,923 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config45: Local shard non-existent not found - shards present: [default] 11:59:44,25 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$z#-1700004565] 11:59:44,36 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config45 11:59:44,41 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config46 11:59:44,42 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config46 11:59:44,62 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:44,62 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config46: onRemoveShardReplica: RemoveShardReplica [shardName=astronauts, memberName=MemberName{name=member-2}] 11:59:44,63 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config46: Sending RemoveServer message to peer akka://test@127.0.0.1:2565/system/testProbe-92#1209374428 for shard member-2-shard-astronauts-config46 11:59:44,74 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config46: onRemoveShardReplica: RemoveShardReplica [shardName=astronauts, memberName=MemberName{name=member-3}] 11:59:44,74 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config46: A shard replica operation for astronauts is already in progress 11:59:44,181 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config47 11:59:44,182 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config47 11:59:44,182 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config47: Creating local shard: member-1-shard-default-config47 11:59:44,192 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config47: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=false] 11:59:44,192 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config47: Returning NotInitializedException for shard default 11:59:44,293 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$A#-510764384] 11:59:44,305 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-config47 11:59:44,307 AM [main] [INFO] ShardManagerTest - testServerRemovedShardActorRunning starting 11:59:44,311 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config48 11:59:44,312 AM [test-akka.actor.default-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config48 11:59:44,312 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config48: Creating local shard: member-1-shard-default-config48 11:59:44,312 AM [test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config48: Creating local shard: member-1-shard-people-config48 11:59:44,323 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:44,323 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config48 11:59:44,323 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config48 11:59:44,324 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config48: Initializing shard [default] 11:59:44,324 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-default-config48 is initialized 11:59:44,324 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config48: findLocalShard : people 11:59:44,324 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config48: Scheduling 600 ms timer to wait for shard ShardInformation [shardId=member-1-shard-people-config48, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 11:59:44,325 AM [test-shard-dispatcher-16] [INFO] Shard - Shard created : member-1-shard-people-config48, persistent : true 11:59:44,328 AM [test-shard-dispatcher-16] [INFO] Shard - Starting recovery for member-1-shard-people-config48 with journal batch size 1 11:59:44,333 AM [test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$k/member-1-shard-people-config48/member-1-shard-people-config48-notifier#-2011614912 created and ready for shard:member-1-shard-people-config48 11:59:44,333 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$k/member-1-shard-people-config48/shard-member-1:snapshot-read#-946223365] 11:59:44,334 AM [test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config48: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 11:59:44,334 AM [test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-config48: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:59:44,334 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config48: Initializing shard [people] 11:59:44,334 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-people-config48 is initialized 11:59:44,334 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-people-config48 is initialized - notifying 1 OnShardInitialized callbacks 11:59:44,334 AM [test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config48: onLeadershipChange, isLeader: false, hasLeader : false 11:59:44,334 AM [test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config48: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config48 is no longer the leader 11:59:44,334 AM [test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config48 , received role change from null to Follower 11:59:44,334 AM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config48 , registered listener akka://test/user/$$k 11:59:44,334 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config48: findLocalShard : default 11:59:44,334 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config48: Received role changed for member-1-shard-people-config48 from null to Follower 11:59:44,335 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config48 : Sending Shutdown to Shard actor Actor[akka://test/user/member-1-shard-default-config48#-1191211297] with 10000 ms timeout 11:59:44,337 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config48 : Local Shard replica for shard default has been removed 11:59:44,337 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config48: persisting the shard list [people] 11:59:44,338 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config48 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 11:59:44,338 AM [main] [INFO] ShardManagerTest - testServerRemovedShardActorRunning ending 11:59:44,338 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config48: Successfully deleted prior snapshots 11:59:44,439 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/member-1-shard-default-config48#-1191211297] 11:59:44,450 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config48 : Successfully shut down Shard actor Actor[akka://test/user/member-1-shard-default-config48#-1191211297] 11:59:44,450 AM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$k] 11:59:44,451 AM [test-shard-dispatcher-16] [INFO] Shard - Stopping Shard member-1-shard-people-config48 11:59:44,451 AM [test-shard-dispatcher-16] [DEBUG] MessageAssembler - member-1-shard-people-config48 (Follower): Closing 11:59:44,451 AM [test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config48: Aborting 0 pending queued transactions 11:59:44,452 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-config48 11:59:44,456 AM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config49 11:59:44,457 AM [test-akka.actor.default-dispatcher-4] [INFO] ShardManager - Recovery complete : shard-manager-config49 11:59:44,457 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config49: Creating local shard: member-1-shard-default-config49 11:59:44,457 AM [main] [INFO] ShardManager - shard-manager-config49: Received role changed for member-1-shard-default-config49 from null to Follower 11:59:44,457 AM [main] [INFO] ShardManager - shard-manager-config49: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config49, leaderId=member-2-shard-default-config49, leaderPayloadVersion=13] 11:59:44,468 AM [main] [INFO] ShardManager - shard-manager-config49: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://test 11:59:44,468 AM [main] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-config49 with address akka://test/user/shardmanager-config49/member-2-shard-default-config49 11:59:44,468 AM [main] [INFO] ShardManager - shard-manager-config49: All Shards are ready - data store config49 is ready 11:59:44,569 AM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$l] 11:59:44,580 AM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config49 11:59:44,583 AM [main] [INFO] ShardManagerTest - testRestoreFromSnapshot starting 11:59:44,585 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config50 11:59:44,587 AM [test-akka.actor.default-dispatcher-4] [INFO] ShardManager - Recovery complete : shard-manager-config50 11:59:44,587 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config50: Restoring from ShardManagerSnapshot: ShardManagerSnapshot [ShardList = [shard1, shard2, astronauts] ] 11:59:44,587 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config50: onSnapshotOffer: ShardManagerSnapshot [ShardList = [shard1, shard2, astronauts] ] 11:59:44,587 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config50: adding shard shard1 11:59:44,587 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config50: adding shard shard2 11:59:44,587 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config50: adding shard astronauts 11:59:44,587 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config50: Creating local shard: member-1-shard-shard2-config50 11:59:44,588 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config50: Creating local shard: member-1-shard-astronauts-config50 11:59:44,588 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config50: Creating local shard: member-1-shard-shard1-config50 11:59:44,588 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:44,588 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-shard2-config50 11:59:44,589 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-astronauts-config50 11:59:44,589 AM [test-shard-dispatcher-16] [INFO] Shard - Shard created : member-1-shard-shard2-config50, persistent : true 11:59:44,589 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-shard1-config50 11:59:44,590 AM [test-shard-dispatcher-16] [INFO] Shard - Starting recovery for member-1-shard-shard2-config50 with journal batch size 1 11:59:44,590 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$m/member-1-shard-shard2-config50/shard-member-1:snapshot-read#624445904] 11:59:44,590 AM [test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$m/member-1-shard-shard2-config50/member-1-shard-shard2-config50-notifier#-1945910211 created and ready for shard:member-1-shard-shard2-config50 11:59:44,590 AM [test-shard-dispatcher-17] [INFO] Shard - Shard created : member-1-shard-astronauts-config50, persistent : true 11:59:44,590 AM [test-shard-dispatcher-16] [INFO] Shard - Shard created : member-1-shard-shard1-config50, persistent : true 11:59:44,591 AM [test-shard-dispatcher-16] [INFO] Shard - Starting recovery for member-1-shard-shard1-config50 with journal batch size 1 11:59:44,591 AM [test-shard-dispatcher-17] [INFO] Shard - Starting recovery for member-1-shard-astronauts-config50 with journal batch size 1 11:59:44,592 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$m/member-1-shard-astronauts-config50/shard-member-1:snapshot-read#-1049759770] 11:59:44,592 AM [test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$m/member-1-shard-astronauts-config50/member-1-shard-astronauts-config50-notifier#1616786556 created and ready for shard:member-1-shard-astronauts-config50 11:59:44,592 AM [test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$m/member-1-shard-shard1-config50/shard-member-1:snapshot-read#1956800984] 11:59:44,593 AM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$m/member-1-shard-shard1-config50/member-1-shard-shard1-config50-notifier#1611836964 created and ready for shard:member-1-shard-shard1-config50 11:59:44,594 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config50: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 11:59:44,594 AM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-shard2-config50: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:59:44,594 AM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard2-config50: onLeadershipChange, isLeader: false, hasLeader : false 11:59:44,594 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config50: onStateChanged: Closing all transaction chains because shard member-1-shard-shard2-config50 is no longer the leader 11:59:44,594 AM [test-akka.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config50: Initializing shard [shard2] 11:59:44,594 AM [test-akka.actor.default-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-shard2-config50 is initialized 11:59:44,594 AM [test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config50 , received role change from null to Follower 11:59:44,594 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config50 (Follower): Received TimeoutNow - switching to Candidate 11:59:44,594 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config50 (Candidate): Election: Candidate has following voting peers: [] 11:59:44,594 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config50: Set currentTerm=1, votedFor=member-1-shard-shard2-config50 11:59:44,594 AM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-shard2-config50 (Candidate): Starting new election term 1 11:59:44,594 AM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-shard2-config50 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:59:44,594 AM [test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-1-shard-shard2-config50 (Follower): Closing 11:59:44,594 AM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard2-config50: onLeadershipChange, isLeader: false, hasLeader : false 11:59:44,594 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config50: onStateChanged: Closing all transaction chains because shard member-1-shard-shard2-config50 is no longer the leader 11:59:44,594 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config50: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 11:59:44,594 AM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-shard1-config50: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:59:44,594 AM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard1-config50: onLeadershipChange, isLeader: false, hasLeader : false 11:59:44,594 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config50: onStateChanged: Closing all transaction chains because shard member-1-shard-shard1-config50 is no longer the leader 11:59:44,594 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config50 (Follower): Received TimeoutNow - switching to Candidate 11:59:44,594 AM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config50 , received role change from Follower to Candidate 11:59:44,594 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config50 (Candidate): Election: Candidate has following voting peers: [] 11:59:44,594 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config50: Set currentTerm=1, votedFor=member-1-shard-shard1-config50 11:59:44,595 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config50: Initializing shard [shard1] 11:59:44,595 AM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-shard1-config50 (Candidate): Starting new election term 1 11:59:44,595 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-shard1-config50 is initialized 11:59:44,595 AM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-shard1-config50 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:59:44,595 AM [test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-1-shard-shard1-config50 (Follower): Closing 11:59:44,595 AM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config50 , received role change from null to Follower 11:59:44,595 AM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard1-config50: onLeadershipChange, isLeader: false, hasLeader : false 11:59:44,595 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config50: onStateChanged: Closing all transaction chains because shard member-1-shard-shard1-config50 is no longer the leader 11:59:44,595 AM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config50 , received role change from Follower to Candidate 11:59:44,595 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 11:59:44,595 AM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-astronauts-config50: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:59:44,595 AM [test-akka.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config50: Initializing shard [astronauts] 11:59:44,595 AM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config50: onLeadershipChange, isLeader: false, hasLeader : false 11:59:44,595 AM [test-akka.actor.default-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-astronauts-config50 is initialized 11:59:44,595 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50: onStateChanged: Closing all transaction chains because shard member-1-shard-astronauts-config50 is no longer the leader 11:59:44,595 AM [test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config50 , received role change from null to Follower 11:59:44,595 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50 (Follower): Received TimeoutNow - switching to Candidate 11:59:44,595 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50 (Candidate): Election: Candidate has following voting peers: [] 11:59:44,595 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50: Set currentTerm=1, votedFor=member-1-shard-astronauts-config50 11:59:44,595 AM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-astronauts-config50 (Candidate): Starting new election term 1 11:59:44,595 AM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-astronauts-config50 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:59:44,595 AM [test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-1-shard-astronauts-config50 (Follower): Closing 11:59:44,595 AM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config50: onLeadershipChange, isLeader: false, hasLeader : false 11:59:44,595 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50: onStateChanged: Closing all transaction chains because shard member-1-shard-astronauts-config50 is no longer the leader 11:59:44,595 AM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config50 , received role change from Follower to Candidate 11:59:44,597 AM [test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-shard2-config50 (Candidate): Received ElectionTimeout 11:59:44,597 AM [test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-shard2-config50 (Leader): Election: Leader has following peers: [] 11:59:44,597 AM [test-shard-dispatcher-16] [INFO] Shard - member-1-shard-shard2-config50 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 11:59:44,597 AM [test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard2-config50: onLeadershipChange, isLeader: true, hasLeader : true 11:59:44,597 AM [test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-shard2-config50: became leader with frontend state for [] 11:59:44,597 AM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config50 , received role change from Candidate to Leader 11:59:44,597 AM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config50 , registered listener akka://test/user/$$m 11:59:44,597 AM [test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-shard1-config50 (Candidate): Received ElectionTimeout 11:59:44,597 AM [test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-shard1-config50 (Leader): Election: Leader has following peers: [] 11:59:44,597 AM [test-shard-dispatcher-16] [INFO] Shard - member-1-shard-shard1-config50 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 11:59:44,597 AM [test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard1-config50: onLeadershipChange, isLeader: true, hasLeader : true 11:59:44,597 AM [test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-shard1-config50: became leader with frontend state for [] 11:59:44,597 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config50: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-shard2-config50, leaderId=member-1-shard-shard2-config50, leaderPayloadVersion=13] 11:59:44,597 AM [test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-astronauts-config50 (Candidate): Received ElectionTimeout 11:59:44,597 AM [test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-astronauts-config50 (Leader): Election: Leader has following peers: [] 11:59:44,597 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config50: Received role changed for member-1-shard-shard2-config50 from Candidate to Leader 11:59:44,597 AM [test-shard-dispatcher-16] [INFO] Shard - member-1-shard-astronauts-config50 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 11:59:44,597 AM [test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config50: onLeadershipChange, isLeader: true, hasLeader : true 11:59:44,597 AM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config50 , received role change from Candidate to Leader 11:59:44,597 AM [test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-astronauts-config50: became leader with frontend state for [] 11:59:44,597 AM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config50 , registered listener akka://test/user/$$m 11:59:44,598 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config50: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-shard1-config50, leaderId=member-1-shard-shard1-config50, leaderPayloadVersion=13] 11:59:44,598 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config50: Received role changed for member-1-shard-shard1-config50 from Candidate to Leader 11:59:44,598 AM [test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config50 , received role change from Candidate to Leader 11:59:44,598 AM [test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config50 , registered listener akka://test/user/$$m 11:59:44,598 AM [test-akka.actor.default-dispatcher-21] [INFO] ShardManager - shard-manager-config50: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-astronauts-config50, leaderId=member-1-shard-astronauts-config50, leaderPayloadVersion=13] 11:59:44,598 AM [test-akka.actor.default-dispatcher-21] [INFO] ShardManager - shard-manager-config50: Received role changed for member-1-shard-astronauts-config50 from Candidate to Leader 11:59:44,598 AM [test-akka.actor.default-dispatcher-21] [INFO] ShardManager - shard-manager-config50: All Shards are ready - data store config50 is ready 11:59:44,598 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config50: findLocalShard : shard1 11:59:44,599 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config50: findLocalShard : shard2 11:59:44,599 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config50: findLocalShard : astronauts 11:59:44,599 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config50: onGetSnapshot 11:59:44,600 AM [test-akka.actor.default-dispatcher-19] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config50: Expecting 3 shard snapshot replies 11:59:44,600 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50: onGetSnapshot 11:59:44,600 AM [test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-shard2-config50: onGetSnapshot 11:59:44,600 AM [test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-shard2-config50: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 11:59:44,600 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 11:59:44,600 AM [test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-shard1-config50: onGetSnapshot 11:59:44,600 AM [test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-shard1-config50: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 11:59:44,606 AM [test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-shard2-config50: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 11:59:44,606 AM [test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-shard1-config50: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 11:59:44,606 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config50: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 11:59:44,606 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config50: Received GetSnapshotReply [id=member-1-shard-shard1-config50, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@8f92b12, electionTerm=1, electionVotedFor=member-1-shard-shard1-config50, ServerConfigPayload=null]] 11:59:44,606 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config50: Received GetSnapshotReply [id=member-1-shard-shard2-config50, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@7f3bd856, electionTerm=1, electionVotedFor=member-1-shard-shard2-config50, ServerConfigPayload=null]] 11:59:44,606 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config50: Received GetSnapshotReply [id=member-1-shard-astronauts-config50, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@2f1ef0d4, electionTerm=1, electionVotedFor=member-1-shard-astronauts-config50, ServerConfigPayload=null]] 11:59:44,606 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config50: All shard snapshots received 11:59:44,607 AM [main] [INFO] ShardManagerTest - testRestoreFromSnapshot ending 11:59:44,709 AM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$m] 11:59:44,720 AM [test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-1-shard-astronauts-config50 11:59:44,720 AM [test-shard-dispatcher-23] [INFO] Shard - Stopping Shard member-1-shard-shard2-config50 11:59:44,720 AM [test-shard-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-shard2-config50 (Leader)_slicer-id-161: Closing 11:59:44,720 AM [test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-astronauts-config50 (Leader)_slicer-id-163: Closing 11:59:44,720 AM [test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-astronauts-config50: Aborting 0 pending queued transactions 11:59:44,720 AM [test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-shard2-config50: Aborting 0 pending queued transactions 11:59:44,721 AM [test-shard-dispatcher-18] [INFO] Shard - Stopping Shard member-1-shard-shard1-config50 11:59:44,721 AM [test-shard-dispatcher-18] [DEBUG] MessageSlicer - member-1-shard-shard1-config50 (Leader)_slicer-id-162: Closing 11:59:44,721 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config50: Aborting 0 pending queued transactions 11:59:44,721 AM [test-akka.actor.default-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-config50 11:59:44,724 AM [main] [INFO] ShardManagerTest - testOnCreateShardWithNoInitialSchemaContext starting 11:59:44,727 AM [test-akka.actor.default-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-config51 11:59:44,728 AM [test-akka.actor.default-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-config51 11:59:44,738 AM [test-akka.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config51: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=foo-ns, moduleName=foo-module, shardName=foo, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@63074751] 11:59:44,738 AM [test-akka.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config51 doCreateShard: shardId: member-1-shard-foo-config51, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: true 11:59:44,738 AM [test-akka.actor.default-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:44,738 AM [test-akka.actor.default-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-foo-config51 11:59:44,739 AM [test-akka.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config51: findLocalShard : foo 11:59:44,739 AM [test-akka.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config51: Scheduling 600 ms timer to wait for shard ShardInformation [shardId=member-1-shard-foo-config51, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 11:59:44,740 AM [test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-foo-config51, persistent : true 11:59:44,740 AM [test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$B/member-1-shard-foo-config51/member-1-shard-foo-config51-notifier#-1610818962 created and ready for shard:member-1-shard-foo-config51 11:59:44,740 AM [test-akka.actor.default-dispatcher-20] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$B/member-1-shard-foo-config51/shard-member-1:snapshot-read#-787726283] 11:59:44,741 AM [test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-foo-config51 with journal batch size 1 11:59:44,742 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-foo-config51: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 11:59:44,742 AM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-foo-config51: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:59:44,742 AM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-foo-config51: onLeadershipChange, isLeader: false, hasLeader : false 11:59:44,742 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-foo-config51: onStateChanged: Closing all transaction chains because shard member-1-shard-foo-config51 is no longer the leader 11:59:44,742 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-foo-config51 (Follower): Received TimeoutNow - switching to Candidate 11:59:44,742 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-foo-config51 (Candidate): Election: Candidate has following voting peers: [] 11:59:44,742 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-foo-config51: Set currentTerm=1, votedFor=member-1-shard-foo-config51 11:59:44,742 AM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-foo-config51 (Candidate): Starting new election term 1 11:59:44,742 AM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-foo-config51 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:59:44,742 AM [test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-1-shard-foo-config51 (Follower): Closing 11:59:44,742 AM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-foo-config51: onLeadershipChange, isLeader: false, hasLeader : false 11:59:44,742 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-foo-config51: onStateChanged: Closing all transaction chains because shard member-1-shard-foo-config51 is no longer the leader 11:59:44,743 AM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config51: Initializing shard [foo] 11:59:44,743 AM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-foo-config51 is initialized 11:59:44,743 AM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-foo-config51 is initialized - notifying 1 OnShardInitialized callbacks 11:59:44,743 AM [test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config51 , received role change from null to Follower 11:59:44,743 AM [test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config51 , received role change from Follower to Candidate 11:59:44,744 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-foo-config51 (Candidate): Received ElectionTimeout 11:59:44,744 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-foo-config51 (Leader): Election: Leader has following peers: [] 11:59:44,744 AM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-foo-config51 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 11:59:44,744 AM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-foo-config51: onLeadershipChange, isLeader: true, hasLeader : true 11:59:44,744 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-foo-config51: became leader with frontend state for [] 11:59:44,744 AM [test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config51 , received role change from Candidate to Leader 11:59:44,744 AM [test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config51 , registered listener akka://test/user/$B 11:59:44,744 AM [test-akka.actor.default-dispatcher-19] [INFO] ShardManager - shard-manager-config51: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-foo-config51, leaderId=member-1-shard-foo-config51, leaderPayloadVersion=13] 11:59:44,744 AM [test-akka.actor.default-dispatcher-19] [INFO] ShardManager - shard-manager-config51: Received role changed for member-1-shard-foo-config51 from Candidate to Leader 11:59:44,744 AM [test-akka.actor.default-dispatcher-19] [INFO] ShardManager - shard-manager-config51: All Shards are ready - data store config51 is ready 11:59:44,744 AM [main] [INFO] ShardManagerTest - testOnCreateShardWithNoInitialSchemaContext ending 11:59:44,846 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$B#-2020968673] 11:59:44,857 AM [test-shard-dispatcher-18] [INFO] Shard - Stopping Shard member-1-shard-foo-config51 11:59:44,858 AM [test-shard-dispatcher-18] [DEBUG] MessageSlicer - member-1-shard-foo-config51 (Leader)_slicer-id-165: Closing 11:59:44,858 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-foo-config51: Aborting 0 pending queued transactions 11:59:44,858 AM [test-akka.actor.default-dispatcher-19] [INFO] ShardManager - Stopping ShardManager shard-manager-config51 11:59:44,860 AM [main] [INFO] ShardManagerTest - testGetSnapshot starting 11:59:44,864 AM [test-akka.actor.default-dispatcher-19] [INFO] ShardManager - Starting ShardManager shard-manager-config52 11:59:44,865 AM [test-akka.actor.default-dispatcher-19] [INFO] ShardManager - Recovery complete : shard-manager-config52 11:59:44,865 AM [test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-config52: Creating local shard: member-1-shard-shard2-config52 11:59:44,865 AM [test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-config52: Creating local shard: member-1-shard-shard1-config52 11:59:44,874 AM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config52: onGetSnapshot 11:59:44,874 AM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:44,874 AM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-shard2-config52 11:59:44,875 AM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-shard1-config52 11:59:44,875 AM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config52: findLocalShard : shard1 11:59:44,875 AM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config52: Scheduling 600 ms timer to wait for shard ShardInformation [shardId=member-1-shard-shard1-config52, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 11:59:44,875 AM [test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-shard2-config52, persistent : true 11:59:44,876 AM [test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$n/member-1-shard-shard2-config52/member-1-shard-shard2-config52-notifier#1599145645 created and ready for shard:member-1-shard-shard2-config52 11:59:44,876 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$n/member-1-shard-shard2-config52/shard-member-1:snapshot-read#-997126839] 11:59:44,876 AM [test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-shard2-config52 with journal batch size 1 11:59:44,877 AM [test-shard-dispatcher-22] [INFO] Shard - Shard created : member-1-shard-shard1-config52, persistent : true 11:59:44,878 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config52: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 11:59:44,878 AM [test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$n/member-1-shard-shard1-config52/member-1-shard-shard1-config52-notifier#452941469 created and ready for shard:member-1-shard-shard1-config52 11:59:44,878 AM [test-akka.actor.default-dispatcher-20] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$n/member-1-shard-shard1-config52/shard-member-1:snapshot-read#690036244] 11:59:44,878 AM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-shard2-config52: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:59:44,878 AM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config52: Initializing shard [shard2] 11:59:44,878 AM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-shard2-config52 is initialized 11:59:44,878 AM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard2-config52: onLeadershipChange, isLeader: false, hasLeader : false 11:59:44,878 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-shard2-config52 is no longer the leader 11:59:44,878 AM [test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config52 , received role change from null to Follower 11:59:44,878 AM [test-shard-dispatcher-22] [INFO] Shard - Starting recovery for member-1-shard-shard1-config52 with journal batch size 1 11:59:44,878 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config52 (Follower): Received TimeoutNow - switching to Candidate 11:59:44,878 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config52 (Candidate): Election: Candidate has following voting peers: [] 11:59:44,878 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config52: Set currentTerm=1, votedFor=member-1-shard-shard2-config52 11:59:44,878 AM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-shard2-config52 (Candidate): Starting new election term 1 11:59:44,878 AM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-shard2-config52 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:59:44,878 AM [test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-1-shard-shard2-config52 (Follower): Closing 11:59:44,878 AM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard2-config52: onLeadershipChange, isLeader: false, hasLeader : false 11:59:44,878 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-shard2-config52 is no longer the leader 11:59:44,878 AM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config52 , received role change from Follower to Candidate 11:59:44,879 AM [test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-shard2-config52 (Candidate): Received ElectionTimeout 11:59:44,879 AM [test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-shard2-config52 (Leader): Election: Leader has following peers: [] 11:59:44,879 AM [test-shard-dispatcher-22] [INFO] Shard - member-1-shard-shard2-config52 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 11:59:44,879 AM [test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard2-config52: onLeadershipChange, isLeader: true, hasLeader : true 11:59:44,879 AM [test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-shard2-config52: became leader with frontend state for [] 11:59:44,880 AM [test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config52 , received role change from Candidate to Leader 11:59:44,880 AM [test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-shard1-config52: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 11:59:44,880 AM [test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard2-config52 , registered listener akka://test/user/$$n 11:59:44,880 AM [test-shard-dispatcher-22] [INFO] Shard - member-1-shard-shard1-config52: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:59:44,880 AM [test-akka.actor.default-dispatcher-19] [INFO] ShardManager - shard-manager-config52: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-shard2-config52, leaderId=member-1-shard-shard2-config52, leaderPayloadVersion=13] 11:59:44,880 AM [test-akka.actor.default-dispatcher-19] [INFO] ShardManager - shard-manager-config52: Received role changed for member-1-shard-shard2-config52 from Candidate to Leader 11:59:44,880 AM [test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard1-config52: onLeadershipChange, isLeader: false, hasLeader : false 11:59:44,880 AM [test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-shard1-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-shard1-config52 is no longer the leader 11:59:44,880 AM [test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-config52: Initializing shard [shard1] 11:59:44,880 AM [test-akka.actor.default-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-shard1-config52 is initialized 11:59:44,880 AM [test-akka.actor.default-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-shard1-config52 is initialized - notifying 1 OnShardInitialized callbacks 11:59:44,880 AM [test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-shard1-config52 (Follower): Received TimeoutNow - switching to Candidate 11:59:44,880 AM [test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config52 , received role change from null to Follower 11:59:44,880 AM [test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-shard1-config52 (Candidate): Election: Candidate has following voting peers: [] 11:59:44,880 AM [test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-shard1-config52: Set currentTerm=1, votedFor=member-1-shard-shard1-config52 11:59:44,880 AM [test-shard-dispatcher-22] [INFO] Shard - member-1-shard-shard1-config52 (Candidate): Starting new election term 1 11:59:44,880 AM [test-shard-dispatcher-22] [INFO] Shard - member-1-shard-shard1-config52 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:59:44,880 AM [test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-1-shard-shard1-config52 (Follower): Closing 11:59:44,880 AM [test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard1-config52: onLeadershipChange, isLeader: false, hasLeader : false 11:59:44,880 AM [test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-shard1-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-shard1-config52 is no longer the leader 11:59:44,880 AM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config52: findLocalShard : shard2 11:59:44,880 AM [test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config52 , received role change from Follower to Candidate 11:59:44,881 AM [test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-shard1-config52 (Candidate): Received ElectionTimeout 11:59:44,881 AM [test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-shard1-config52 (Leader): Election: Leader has following peers: [] 11:59:44,881 AM [test-shard-dispatcher-22] [INFO] Shard - member-1-shard-shard1-config52 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 11:59:44,881 AM [test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-shard1-config52: onLeadershipChange, isLeader: true, hasLeader : true 11:59:44,881 AM [test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-shard1-config52: became leader with frontend state for [] 11:59:44,881 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config52: onGetSnapshot 11:59:44,881 AM [test-akka.actor.default-dispatcher-13] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Expecting 2 shard snapshot replies 11:59:44,881 AM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config52 , received role change from Candidate to Leader 11:59:44,881 AM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-shard1-config52 , registered listener akka://test/user/$$n 11:59:44,881 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config52: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-shard1-config52, leaderId=member-1-shard-shard1-config52, leaderPayloadVersion=13] 11:59:44,881 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config52: Received role changed for member-1-shard-shard1-config52 from Candidate to Leader 11:59:44,881 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config52: All Shards are ready - data store config52 is ready 11:59:44,882 AM [test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-shard2-config52: onGetSnapshot 11:59:44,882 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config52: onGetSnapshot 11:59:44,882 AM [test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-shard2-config52: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 11:59:44,882 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config52: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 11:59:44,882 AM [test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-shard2-config52: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 11:59:44,882 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config52: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 11:59:44,882 AM [test-akka.actor.default-dispatcher-19] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Received GetSnapshotReply [id=member-1-shard-shard2-config52, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@e1751c0, electionTerm=1, electionVotedFor=member-1-shard-shard2-config52, ServerConfigPayload=null]] 11:59:44,882 AM [test-akka.actor.default-dispatcher-19] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Received GetSnapshotReply [id=member-1-shard-shard1-config52, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@48d45adf, electionTerm=1, electionVotedFor=member-1-shard-shard1-config52, ServerConfigPayload=null]] 11:59:44,882 AM [test-akka.actor.default-dispatcher-19] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: All shard snapshots received 11:59:44,893 AM [test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-config52: onAddShardReplica: AddShardReplica[ShardName=astronauts] 11:59:44,894 AM [test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-config52: Sending AddServer message to peer akka://test@127.0.0.1:2565/system/testProbe-106#-2115160229 for shard member-1-shard-astronauts-config52 11:59:44,895 AM [test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-astronauts-config52, persistent : true 11:59:44,896 AM [test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-astronauts-config52 with journal batch size 1 11:59:44,896 AM [test-akka.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$$n/member-1-shard-astronauts-config52/shard-member-1:snapshot-read#-1777533081] 11:59:44,897 AM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$$n/member-1-shard-astronauts-config52/member-1-shard-astronauts-config52-notifier#181076001 created and ready for shard:member-1-shard-astronauts-config52 11:59:44,898 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config52: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 11:59:44,898 AM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-astronauts-config52: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:59:44,898 AM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config52: Received AddServerReply{status=OK, leaderHint=} for shard astronauts from leader akka://test@127.0.0.1:2565/system/testProbe-106#-2115160229 11:59:44,898 AM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config52: Leader shard successfully added the replica shard astronauts 11:59:44,898 AM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config52: onLeadershipChange, isLeader: false, hasLeader : false 11:59:44,898 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-astronauts-config52 is no longer the leader 11:59:44,898 AM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config52 , received role change from null to Follower 11:59:44,898 AM [test-shard-dispatcher-18] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:59:44,898 AM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-astronauts-config52 11:59:44,898 AM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config52: persisting the shard list [shard2, astronauts, shard1] 11:59:44,898 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config52: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 11:59:44,898 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config52: Re-initializing to Follower with previous leaderId null 11:59:44,899 AM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config52: Initializing shard [astronauts] 11:59:44,899 AM [test-akka.actor.default-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-astronauts-config52 is initialized 11:59:44,899 AM [test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-1-shard-astronauts-config52 (Follower): Closing 11:59:44,899 AM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config52: onLeadershipChange, isLeader: false, hasLeader : false 11:59:44,899 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-astronauts-config52 is no longer the leader 11:59:44,899 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config52 (Follower): Received TimeoutNow - switching to Candidate 11:59:44,899 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config52 (Candidate): Election: Candidate has following voting peers: [] 11:59:44,899 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config52: Set currentTerm=1, votedFor=member-1-shard-astronauts-config52 11:59:44,899 AM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-astronauts-config52 (Candidate): Starting new election term 1 11:59:44,899 AM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-astronauts-config52 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:59:44,899 AM [test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-1-shard-astronauts-config52 (Follower): Closing 11:59:44,899 AM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config52: onLeadershipChange, isLeader: false, hasLeader : false 11:59:44,899 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config52: onStateChanged: Closing all transaction chains because shard member-1-shard-astronauts-config52 is no longer the leader 11:59:44,899 AM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config52 , received role change from Follower to Candidate 11:59:44,899 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config52 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 11:59:44,899 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config52: Successfully deleted prior snapshots 11:59:44,899 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config52 (Candidate): Received ElectionTimeout 11:59:44,900 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config52 (Leader): Election: Leader has following peers: [] 11:59:44,900 AM [test-shard-dispatcher-18] [INFO] Shard - member-1-shard-astronauts-config52 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 11:59:44,900 AM [test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-astronauts-config52: onLeadershipChange, isLeader: true, hasLeader : true 11:59:44,900 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-astronauts-config52: became leader with frontend state for [] 11:59:44,900 AM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config52 , received role change from Candidate to Leader 11:59:44,900 AM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-astronauts-config52 , registered listener akka://test/user/$$n 11:59:44,900 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config52: findLocalShard : astronauts 11:59:44,900 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config52: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-astronauts-config52, leaderId=member-1-shard-astronauts-config52, leaderPayloadVersion=13] 11:59:44,900 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config52: Received role changed for member-1-shard-astronauts-config52 from Candidate to Leader 11:59:44,900 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config52: All Shards are ready - data store config52 is ready 11:59:44,900 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config52: onGetSnapshot 11:59:44,900 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config52: onGetSnapshot 11:59:44,900 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config52: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 11:59:44,900 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Expecting 3 shard snapshot replies 11:59:44,900 AM [test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-astronauts-config52: onGetSnapshot 11:59:44,900 AM [test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-astronauts-config52: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 11:59:44,901 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard2-config52: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 11:59:44,901 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config52: onGetSnapshot 11:59:44,901 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config52: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 11:59:44,901 AM [test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-astronauts-config52: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 11:59:44,901 AM [test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-shard1-config52: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 11:59:44,901 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Received GetSnapshotReply [id=member-1-shard-shard2-config52, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@619424ff, electionTerm=1, electionVotedFor=member-1-shard-shard2-config52, ServerConfigPayload=null]] 11:59:44,901 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Received GetSnapshotReply [id=member-1-shard-shard1-config52, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@3d564d7b, electionTerm=1, electionVotedFor=member-1-shard-shard1-config52, ServerConfigPayload=null]] 11:59:44,901 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: Received GetSnapshotReply [id=member-1-shard-astronauts-config52, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@7da525f2, electionTerm=1, electionVotedFor=member-1-shard-astronauts-config52, ServerConfigPayload=null]] 11:59:44,901 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config52: All shard snapshots received 11:59:44,902 AM [main] [INFO] ShardManagerTest - testGetSnapshot ending 11:59:45,3 AM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$n] 11:59:45,15 AM [test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-1-shard-shard1-config52 11:59:45,15 AM [test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-shard1-config52 (Leader)_slicer-id-169: Closing 11:59:45,15 AM [test-shard-dispatcher-23] [INFO] Shard - Stopping Shard member-1-shard-astronauts-config52 11:59:45,15 AM [test-shard-dispatcher-17] [INFO] Shard - Stopping Shard member-1-shard-shard2-config52 11:59:45,15 AM [test-shard-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-astronauts-config52 (Leader)_slicer-id-171: Closing 11:59:45,15 AM [test-shard-dispatcher-17] [DEBUG] MessageSlicer - member-1-shard-shard2-config52 (Leader)_slicer-id-168: Closing 11:59:45,15 AM [test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-shard1-config52: Aborting 0 pending queued transactions 11:59:45,15 AM [test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-shard2-config52: Aborting 0 pending queued transactions 11:59:45,15 AM [test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-astronauts-config52: Aborting 0 pending queued transactions 11:59:45,15 AM [test-akka.actor.default-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-config52 11:59:45,17 AM [main] [INFO] ShardManagerTest - testWhenMultipleShardsPresentSyncStatusMustBeTrueForAllShards starting 11:59:45,21 AM [main] [INFO] ShardManager - Starting ShardManager shard-manager-config53 11:59:45,22 AM [test-akka.actor.default-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-config53 11:59:45,22 AM [test-akka.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config53: Creating local shard: member-1-shard-default-config53 11:59:45,23 AM [test-akka.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config53: Creating local shard: member-1-shard-astronauts-config53 11:59:45,23 AM [main] [INFO] ShardManager - shard-manager-config53: Received role changed for member-1-shard-default-config53 from Follower to Leader 11:59:45,23 AM [main] [INFO] ShardManager - shard-manager-config53: Received role changed for member-1-shard-astronauts-config53 from Follower to Leader 11:59:45,23 AM [main] [INFO] ShardManager - shard-manager-config53: Received role changed for member-1-shard-astronauts-config53 from Leader to Follower 11:59:45,23 AM [main] [INFO] ShardManager - shard-manager-config53 Received follower initial sync status for member-1-shard-astronauts-config53 status sync done true 11:59:45,23 AM [main] [INFO] ShardManagerTest - testWhenMultipleShardsPresentSyncStatusMustBeTrueForAllShards ending 11:59:45,124 AM [main] [INFO] TestActorFactory - Killing actor TestActor[akka://test/user/$$o] 11:59:45,135 AM [main] [INFO] ShardManager - Stopping ShardManager shard-manager-config53 11:59:45,138 AM [main] [INFO] ShardManagerTest - testAddShardReplicaWithFindPrimaryTimeout starting 11:59:45,144 AM [test-akka.actor.default-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-config54 11:59:45,145 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config54 11:59:45,155 AM [test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:45,155 AM [test-akka.actor.default-dispatcher-24] [INFO] ShardManager - shard-manager-config54: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://non-existent@127.0.0.1:5 11:59:45,155 AM [test-akka.actor.default-dispatcher-24] [INFO] ShardManager - shard-manager-config54: All Shards are ready - data store config54 is ready 11:59:45,155 AM [test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-config54: onAddShardReplica: AddShardReplica[ShardName=astronauts] 11:59:45,155 AM [test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-config54: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=astronauts, waitUntilReady=true] 11:59:45,156 AM [test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-config54: findPrimary for astronauts forwarding to remote ShardManager akka://non-existent@127.0.0.1:5/user/shardmanager-config54, visitedAddresses: [akka://test@127.0.0.1:2550/user/shardmanager-config54] 11:59:45,174 AM [test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://non-existent@127.0.0.1:5], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:5,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 11:59:45,175 AM [test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://non-existent@127.0.0.1:5], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:5,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 11:59:45,375 AM [test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-config54: Received failure from FindPrimary for shard astronauts akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://test/user/shardmanager-config54#-2046306098]] after [200 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.FindPrimary]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 11:59:45,375 AM [main] [INFO] ShardManagerTest - testAddShardReplicaWithFindPrimaryTimeout ending 11:59:45,477 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/shardmanager-config54#-2046306098] 11:59:45,488 AM [test-akka.actor.default-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-config54 11:59:45,490 AM [main] [INFO] ShardManagerTest - testOnCreateShard starting 11:59:45,494 AM [test-akka.actor.default-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-config55 11:59:45,495 AM [test-akka.actor.default-dispatcher-24] [INFO] ShardManager - Recovery complete : shard-manager-config55 11:59:45,495 AM [test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 1 11:59:45,504 AM [test-akka.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config55: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=foo-ns, moduleName=foo-module, shardName=foo, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-5}, MemberName{name=member-6}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@48c0366d] 11:59:45,505 AM [test-akka.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config55 doCreateShard: shardId: member-1-shard-foo-config55, memberNames: [MemberName{name=member-1}, MemberName{name=member-5}, MemberName{name=member-6}], peerAddresses: {member-6-shard-foo-config55=null, member-5-shard-foo-config55=null}, isActiveMember: true 11:59:45,505 AM [test-akka.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config55: findLocalShard : foo 11:59:45,505 AM [test-akka.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config55: Scheduling 300000 ms timer to wait for shard ShardInformation [shardId=member-1-shard-foo-config55, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 11:59:45,507 AM [test-shard-dispatcher-23] [INFO] Shard - Shard created : member-1-shard-foo-config55, persistent : false 11:59:45,508 AM [test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://test/user/$C/member-1-shard-foo-config55/shard-member-1:snapshot-read#-957933768] 11:59:45,509 AM [test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://test@127.0.0.1:2565/user/$C/member-1-shard-foo-config55/member-1-shard-foo-config55-notifier#-786944007 created and ready for shard:member-1-shard-foo-config55 11:59:45,509 AM [test-shard-dispatcher-23] [INFO] Shard - Starting recovery for member-1-shard-foo-config55 with journal batch size 1 11:59:45,511 AM [test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-foo-config55: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://test/deadLetters] 11:59:45,511 AM [test-shard-dispatcher-23] [INFO] Shard - member-1-shard-foo-config55: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:59:45,512 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config55: Initializing shard [foo] 11:59:45,512 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-foo-config55 is initialized 11:59:45,512 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-foo-config55 is initialized - notifying 1 OnShardInitialized callbacks 11:59:45,512 AM [test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-foo-config55: onLeadershipChange, isLeader: false, hasLeader : false 11:59:45,512 AM [test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-foo-config55: onStateChanged: Closing all transaction chains because shard member-1-shard-foo-config55 is no longer the leader 11:59:45,512 AM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config55 , received role change from null to Follower 11:59:45,512 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config55: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=foo-ns, moduleName=foo-module, shardName=foo, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-5}, MemberName{name=member-6}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@48c0366d] 11:59:45,512 AM [test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config55: Shard foo already exists 11:59:45,512 AM [test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-foo-config55 , registered listener akka://test/user/$C 11:59:45,512 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - shard-manager-config55: Received role changed for member-1-shard-foo-config55 from null to Follower 11:59:45,513 AM [main] [INFO] ShardManagerTest - testOnCreateShard ending 11:59:45,614 AM [main] [INFO] TestActorFactory - Killing actor Actor[akka://test/user/$C#-1032432260] 11:59:45,625 AM [test-shard-dispatcher-23] [INFO] Shard - Stopping Shard member-1-shard-foo-config55 11:59:45,625 AM [test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-1-shard-foo-config55 (Follower): Closing 11:59:45,626 AM [test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-foo-config55: Aborting 0 pending queued transactions 11:59:45,626 AM [test-akka.actor.default-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-config55 11:59:45,627 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config1 11:59:45,627 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config43 11:59:45,627 AM [test-akka.actor.default-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-config35 11:59:45,627 AM [test-akka.actor.default-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config21 11:59:45,627 AM [test-akka.actor.default-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-config11 11:59:45,627 AM [test-akka.actor.default-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-config46 11:59:45,628 AM [test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:59:45,628 AM [test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:59:45,633 AM [test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/26/2024 11:59:45.634] [test-akka.actor.internal-dispatcher-8] [Cluster(akka://test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config